[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:08:31 dist-test-slave-l8h4 krb5kdc[5493](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:08:31 dist-test-slave-l8h4 krb5kdc[5493](info): set up 2 sockets
Jun 20 20:08:31 dist-test-slave-l8h4 krb5kdc[5493](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260620 20:08:32.130796 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.955s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:08:32 dist-test-slave-l8h4 krb5kdc[5493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:08:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:08:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:08:32.254767 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:39033
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:39033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:32.371219 5506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:32.371510 5506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:32.371568 5506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:32.376010 5506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:08:32.376102 5506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:32.376133 5506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:32.376160 5506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:08:32.376195 5506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:08:32.382211 5506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:39033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:39033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:32.383358 5506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:32.384434 5506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:32.393378 5512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:32.393523 5506 server_base.cc:1034] running on GCE node
W20260620 20:08:32.393384 5514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:32.393385 5511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:32.394444 5506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:32.395694 5506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:32.396917 5506 hybrid_clock.cc:648] HybridClock initialized: now 1781986112396893 us; error 37 us; skew 500 ppm
Jun 20 20:08:32 dist-test-slave-l8h4 krb5kdc[5493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986112, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:32.399669 5506 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:08:32.400772 5506 webserver.cc:460] Webserver started at http://127.5.91.190:44605/ using document root <none> and password file <none>
I20260620 20:08:32.401360 5506 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:32.401429 5506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:32.401625 5506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:32.403746 5506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "7850037604a34676abcb3490666c56c7"
format_stamp: "Formatted at 2026-06-20 20:08:32 on dist-test-slave-l8h4"
server_key: "48927a204d65815f064fd5bcda506474"
server_key_iv: "eeed3a07e36f8ac62acd58988f2a5e6e"
server_key_version: "encryptionkey@0"
I20260620 20:08:32.404330 5506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "7850037604a34676abcb3490666c56c7"
format_stamp: "Formatted at 2026-06-20 20:08:32 on dist-test-slave-l8h4"
server_key: "48927a204d65815f064fd5bcda506474"
server_key_iv: "eeed3a07e36f8ac62acd58988f2a5e6e"
server_key_version: "encryptionkey@0"
I20260620 20:08:32.408252 5506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260620 20:08:32.410928 5521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:32.412216 5506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:08:32.412354 5506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "7850037604a34676abcb3490666c56c7"
format_stamp: "Formatted at 2026-06-20 20:08:32 on dist-test-slave-l8h4"
server_key: "48927a204d65815f064fd5bcda506474"
server_key_iv: "eeed3a07e36f8ac62acd58988f2a5e6e"
server_key_version: "encryptionkey@0"
I20260620 20:08:32.412459 5506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:32.447844 5506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:32.452534 5506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:32.453091 5506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:32.460764 5506 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:39033
I20260620 20:08:32.460781 5573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:39033 every 8 connection(s)
I20260620 20:08:32.461812 5506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:08:32.466100 5574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:32.472182 5574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7850037604a34676abcb3490666c56c7: Bootstrap starting.
I20260620 20:08:32.472498 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 5506
I20260620 20:08:32.472657 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:08:32.474315 5486 external_mini_cluster.cc:1428] Setting key 62b8500a674fab752c65ff96f07a4e5e
I20260620 20:08:32.475572 5574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7850037604a34676abcb3490666c56c7: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:32.476547 5574 log.cc:824] T 00000000000000000000000000000000 P 7850037604a34676abcb3490666c56c7: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:32.479707 5574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7850037604a34676abcb3490666c56c7: No bootstrap required, opened a new log
I20260620 20:08:32.483876 5574 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7850037604a34676abcb3490666c56c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7850037604a34676abcb3490666c56c7" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39033 } }
I20260620 20:08:32.484269 5574 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7850037604a34676abcb3490666c56c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:32.484342 5574 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7850037604a34676abcb3490666c56c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7850037604a34676abcb3490666c56c7, State: Initialized, Role: FOLLOWER
I20260620 20:08:32.484875 5574 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7850037604a34676abcb3490666c56c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7850037604a34676abcb3490666c56c7" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39033 } }
I20260620 20:08:32.485034 5574 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7850037604a34676abcb3490666c56c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:08:32.485091 5574 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7850037604a34676abcb3490666c56c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:08:32.485188 5574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7850037604a34676abcb3490666c56c7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:32.486124 5574 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7850037604a34676abcb3490666c56c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7850037604a34676abcb3490666c56c7" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39033 } }
I20260620 20:08:32.486500 5574 leader_election.cc:302] T 00000000000000000000000000000000 P 7850037604a34676abcb3490666c56c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7850037604a34676abcb3490666c56c7; no voters:
I20260620 20:08:32.486794 5574 leader_election.cc:288] T 00000000000000000000000000000000 P 7850037604a34676abcb3490666c56c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:08:32.486882 5579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7850037604a34676abcb3490666c56c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:32.487049 5579 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7850037604a34676abcb3490666c56c7 [term 1 LEADER]: Becoming Leader. State: Replica: 7850037604a34676abcb3490666c56c7, State: Running, Role: LEADER
I20260620 20:08:32.487351 5579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7850037604a34676abcb3490666c56c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7850037604a34676abcb3490666c56c7" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39033 } }
I20260620 20:08:32.488134 5574 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7850037604a34676abcb3490666c56c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:08:32.489226 5580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7850037604a34676abcb3490666c56c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7850037604a34676abcb3490666c56c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7850037604a34676abcb3490666c56c7" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39033 } } }
I20260620 20:08:32.489370 5580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7850037604a34676abcb3490666c56c7 [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:32.489614 5581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7850037604a34676abcb3490666c56c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7850037604a34676abcb3490666c56c7. Latest consensus state: current_term: 1 leader_uuid: "7850037604a34676abcb3490666c56c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7850037604a34676abcb3490666c56c7" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39033 } } }
I20260620 20:08:32.489710 5581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7850037604a34676abcb3490666c56c7 [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:32.489989 5588 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:08:32.493454 5588 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 20:08:32 dist-test-slave-l8h4 krb5kdc[5493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:32.500151 5588 catalog_manager.cc:1347] Generated new cluster ID: bdd9fbe96d6547558f2ebd71393a60c3
I20260620 20:08:32.500222 5588 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:08:32.504671 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:32.478385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42170 (local address 127.5.91.190:39033)
0620 20:08:32.478953 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:32.478976 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:32.479996 (+ 1020us) server_negotiation.cc:407] Connection header received
0620 20:08:32.483371 (+ 3375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:32.483396 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:32.483824 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:32.484368 (+ 544us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:32.486746 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:32.488099 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:32.490218 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:32.490938 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:32.491098 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:32.498552 (+ 7454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:32.498561 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:32.498576 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:32.498606 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:32.500466 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:32.501163 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:32.501169 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:32.501175 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:32.501311 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:32.501836 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:32.501843 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:32.501845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:32.502613 (+ 768us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:32.502836 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:32.503476 (+ 640us) server_negotiation.cc:299] Negotiation successful
0620 20:08:32.503759 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":137,"threads_started":1}
I20260620 20:08:32.508685 5588 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:08:32.509594 5588 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:08:32.516721 5588 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7850037604a34676abcb3490666c56c7: Generated new TSK 0
I20260620 20:08:32.517787 5588 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.91.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
I20260620 20:08:32.554291 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:39033
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:32.673617 5600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:32.674003 5600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:32.674073 5600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:32.678974 5600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:32.679060 5600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:32.679126 5600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:08:32.684875 5600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:39033
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:32.686605 5600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:32.688119 5600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:32.700117 5605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:32.700136 5608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:32.700268 5600 server_base.cc:1034] running on GCE node
W20260620 20:08:32.700630 5606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:32.701108 5600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:32.701797 5600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:32.705155 5600 hybrid_clock.cc:648] HybridClock initialized: now 1781986112705059 us; error 89 us; skew 500 ppm
Jun 20 20:08:32 dist-test-slave-l8h4 krb5kdc[5493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986112, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:32.708694 5600 init.cc:377] Logged in from keytab as kudu/127.5.91.129@KRBTEST.COM (short username kudu)
I20260620 20:08:32.709622 5600 webserver.cc:460] Webserver started at http://127.5.91.129:46465/ using document root <none> and password file <none>
I20260620 20:08:32.710194 5600 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:32.710268 5600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:32.710469 5600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:32.712530 5600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "b1fe76170e9b43458304d843ea29a3cb"
format_stamp: "Formatted at 2026-06-20 20:08:32 on dist-test-slave-l8h4"
server_key: "0912e5c65090583f874e81c582d87028"
server_key_iv: "ad1dbee6336a69def6335317e331bdab"
server_key_version: "encryptionkey@0"
I20260620 20:08:32.713032 5600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "b1fe76170e9b43458304d843ea29a3cb"
format_stamp: "Formatted at 2026-06-20 20:08:32 on dist-test-slave-l8h4"
server_key: "0912e5c65090583f874e81c582d87028"
server_key_iv: "ad1dbee6336a69def6335317e331bdab"
server_key_version: "encryptionkey@0"
I20260620 20:08:32.717530 5600 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 20:08:32.721174 5615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:32.722918 5600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:08:32.723050 5600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "b1fe76170e9b43458304d843ea29a3cb"
format_stamp: "Formatted at 2026-06-20 20:08:32 on dist-test-slave-l8h4"
server_key: "0912e5c65090583f874e81c582d87028"
server_key_iv: "ad1dbee6336a69def6335317e331bdab"
server_key_version: "encryptionkey@0"
I20260620 20:08:32.723166 5600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:32.738652 5600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:32.742280 5600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:32.742446 5600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:32.743031 5600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:08:32.744220 5600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:32.744285 5600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:32.744328 5600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:32.744347 5600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:32.753373 5600 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:39261
I20260620 20:08:32.753466 5728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:39261 every 8 connection(s)
I20260620 20:08:32.754309 5600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:08:32.761622 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 5600
I20260620 20:08:32.761721 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:08:32.761960 5486 external_mini_cluster.cc:1428] Setting key 2338cfec7aba7215ad64abefa8f25a02
Jun 20 20:08:32 dist-test-slave-l8h4 krb5kdc[5493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986112, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:32.771399 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:32.756325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:36033 (local address 127.5.91.190:39033)
0620 20:08:32.756731 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:32.756736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:32.757446 (+ 710us) server_negotiation.cc:407] Connection header received
0620 20:08:32.759459 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:32.759463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:32.759544 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:32.759823 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:32.761265 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:32.761849 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:32.763719 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:32.764015 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:32.764224 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:32.767258 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:32.767262 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:32.767265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:32.767294 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:32.769103 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:32.769714 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:32.769719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:32.769721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:32.769811 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:32.770179 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:32.770182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:32.770184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:32.770418 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:32.770517 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:32.771087 (+ 570us) server_negotiation.cc:299] Negotiation successful
0620 20:08:32.771221 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260620 20:08:32.772163 5731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:32.756663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:39033 (local address 127.5.91.129:36033)
0620 20:08:32.757232 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:32.757282 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:32.759152 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:32.760081 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:32.760090 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:32.760548 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:32.761096 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:32.761111 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:32.761966 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:32.761969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:32.763408 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:32.763425 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:32.764268 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:32.764273 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:32.764486 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:32.765088 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:32.765105 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:32.767017 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:32.769257 (+ 2240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:32.769263 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:32.769280 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:32.769594 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:32.769912 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:32.769914 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:32.769915 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:32.770060 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:32.770548 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:32.770553 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:32.770827 (+ 274us) client_negotiation.cc:769] Sending connection context
0620 20:08:32.770994 (+ 167us) client_negotiation.cc:240] Negotiation successful
0620 20:08:32.771279 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":166,"threads_started":1}
WARNING: no policy specified for kudu/127.5.91.130@KRBTEST.COM; defaulting to no policy
I20260620 20:08:32.773708 5729 heartbeater.cc:347] Connected to a master server at 127.5.91.190:39033
I20260620 20:08:32.773960 5729 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:32.774509 5729 heartbeater.cc:511] Master 127.5.91.190:39033 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.130@KRBTEST.COM" created.
I20260620 20:08:32.776844 5535 ts_manager.cc:194] Registered new tserver with Master: b1fe76170e9b43458304d843ea29a3cb (127.5.91.129:39261)
I20260620 20:08:32.778421 5535 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:36033
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
I20260620 20:08:32.794077 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:39033
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:32.915287 5734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:32.915602 5734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:32.915664 5734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:32.921684 5734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:32.921762 5734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:32.921828 5734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:08:32.929266 5734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:39033
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:32.931030 5734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:32.932562 5734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:32.943113 5739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:32.943135 5742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:32.943200 5740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:32.943630 5734 server_base.cc:1034] running on GCE node
I20260620 20:08:32.944060 5734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:32.944757 5734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:32.945945 5734 hybrid_clock.cc:648] HybridClock initialized: now 1781986112945929 us; error 28 us; skew 500 ppm
Jun 20 20:08:32 dist-test-slave-l8h4 krb5kdc[5493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986112, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:32.949389 5734 init.cc:377] Logged in from keytab as kudu/127.5.91.130@KRBTEST.COM (short username kudu)
I20260620 20:08:32.950498 5734 webserver.cc:460] Webserver started at http://127.5.91.130:41985/ using document root <none> and password file <none>
I20260620 20:08:32.951095 5734 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:32.951164 5734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:32.951363 5734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:32.953430 5734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "3da602126bb54662b3261062ca571371"
format_stamp: "Formatted at 2026-06-20 20:08:32 on dist-test-slave-l8h4"
server_key: "83dc322c1af52bd0e1595f0e83440cc0"
server_key_iv: "e4583e5553bc6b36d23751d7492cb1fb"
server_key_version: "encryptionkey@0"
I20260620 20:08:32.953974 5734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "3da602126bb54662b3261062ca571371"
format_stamp: "Formatted at 2026-06-20 20:08:32 on dist-test-slave-l8h4"
server_key: "83dc322c1af52bd0e1595f0e83440cc0"
server_key_iv: "e4583e5553bc6b36d23751d7492cb1fb"
server_key_version: "encryptionkey@0"
I20260620 20:08:32.958545 5734 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 20:08:32.961102 5749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:32.963287 5734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:08:32.963430 5734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "3da602126bb54662b3261062ca571371"
format_stamp: "Formatted at 2026-06-20 20:08:32 on dist-test-slave-l8h4"
server_key: "83dc322c1af52bd0e1595f0e83440cc0"
server_key_iv: "e4583e5553bc6b36d23751d7492cb1fb"
server_key_version: "encryptionkey@0"
I20260620 20:08:32.963632 5734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:32.978404 5734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:32.981662 5734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:32.981842 5734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:32.982467 5734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:08:32.983305 5734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:32.983352 5734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:32.983405 5734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:32.983431 5734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:32.993881 5734 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:36905
I20260620 20:08:32.993901 5862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:36905 every 8 connection(s)
I20260620 20:08:32.994900 5734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:08:33.002102 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 5734
I20260620 20:08:33.002398 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:08:33.002947 5486 external_mini_cluster.cc:1428] Setting key a9f6180630df01facb737524a96e26ea
Jun 20 20:08:33 dist-test-slave-l8h4 krb5kdc[5493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986112, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:33.013098 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:32.997275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:39857 (local address 127.5.91.190:39033)
0620 20:08:32.997469 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:32.997474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:32.998082 (+ 608us) server_negotiation.cc:407] Connection header received
0620 20:08:32.999688 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:32.999691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:32.999746 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:32.999875 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:33.001324 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:33.002342 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:33.005004 (+ 2662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:33.005325 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:33.005433 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:33.008492 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:33.008496 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:33.008499 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:33.008528 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:33.010107 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:33.011157 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:33.011161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:33.011163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:33.011258 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:33.011822 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:33.011826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:33.011827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:33.012076 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:33.012193 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:33.012752 (+ 559us) server_negotiation.cc:299] Negotiation successful
0620 20:08:33.012903 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
WARNING: no policy specified for kudu/127.5.91.131@KRBTEST.COM; defaulting to no policy
I20260620 20:08:33.013921 5865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:32.997444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:39033 (local address 127.5.91.130:39857)
0620 20:08:32.997931 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:32.997975 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:32.999374 (+ 1399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:33.000081 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:33.000088 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:33.000547 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:33.001059 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:33.001073 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:33.002558 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:33.002562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:33.004501 (+ 1939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:33.004525 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:33.005518 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:33.005524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:33.005722 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:33.006424 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:33.006442 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:33.008270 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:33.010356 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:33.010362 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:33.010379 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:33.010989 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:33.011431 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:33.011434 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:33.011436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:33.011603 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:33.012214 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:33.012220 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:33.012505 (+ 285us) client_negotiation.cc:769] Sending connection context
0620 20:08:33.012677 (+ 172us) client_negotiation.cc:240] Negotiation successful
0620 20:08:33.012980 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":118,"threads_started":1}
I20260620 20:08:33.016342 5863 heartbeater.cc:347] Connected to a master server at 127.5.91.190:39033
I20260620 20:08:33.016645 5863 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:33.017160 5863 heartbeater.cc:511] Master 127.5.91.190:39033 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.131@KRBTEST.COM" created.
I20260620 20:08:33.018673 5535 ts_manager.cc:194] Registered new tserver with Master: 3da602126bb54662b3261062ca571371 (127.5.91.130:36905)
I20260620 20:08:33.019341 5535 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:39857
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
I20260620 20:08:33.038450 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:39033
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:33.157811 5868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:33.158106 5868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:33.158187 5868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:33.163369 5868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:33.163460 5868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:33.163527 5868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:08:33.171372 5868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:39033
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:33.172909 5868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:33.173975 5868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:33.184335 5876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:33.184335 5874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:33.184335 5873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:33.185268 5868 server_base.cc:1034] running on GCE node
I20260620 20:08:33.185899 5868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:33.186820 5868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:33.188036 5868 hybrid_clock.cc:648] HybridClock initialized: now 1781986113187992 us; error 66 us; skew 500 ppm
Jun 20 20:08:33 dist-test-slave-l8h4 krb5kdc[5493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986113, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:33.191972 5868 init.cc:377] Logged in from keytab as kudu/127.5.91.131@KRBTEST.COM (short username kudu)
I20260620 20:08:33.192925 5868 webserver.cc:460] Webserver started at http://127.5.91.131:37291/ using document root <none> and password file <none>
I20260620 20:08:33.193492 5868 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:33.193563 5868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:33.193758 5868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:33.195845 5868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "1fbcafcc449449c4ab63d00b0ba10de6"
format_stamp: "Formatted at 2026-06-20 20:08:33 on dist-test-slave-l8h4"
server_key: "e4fd339ad783dfccdd2607dabdb67931"
server_key_iv: "47f83d11c7297985d8bb940a50a15b11"
server_key_version: "encryptionkey@0"
I20260620 20:08:33.196390 5868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "1fbcafcc449449c4ab63d00b0ba10de6"
format_stamp: "Formatted at 2026-06-20 20:08:33 on dist-test-slave-l8h4"
server_key: "e4fd339ad783dfccdd2607dabdb67931"
server_key_iv: "47f83d11c7297985d8bb940a50a15b11"
server_key_version: "encryptionkey@0"
I20260620 20:08:33.200786 5868 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260620 20:08:33.204028 5883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:33.205830 5868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 20:08:33.205981 5868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "1fbcafcc449449c4ab63d00b0ba10de6"
format_stamp: "Formatted at 2026-06-20 20:08:33 on dist-test-slave-l8h4"
server_key: "e4fd339ad783dfccdd2607dabdb67931"
server_key_iv: "47f83d11c7297985d8bb940a50a15b11"
server_key_version: "encryptionkey@0"
I20260620 20:08:33.206103 5868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:33.232451 5868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:33.236389 5868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:33.236577 5868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:33.237183 5868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:08:33.238016 5868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:33.238061 5868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:33.238113 5868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:33.238164 5868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:33.248186 5868 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:41955
I20260620 20:08:33.248220 5996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:41955 every 8 connection(s)
I20260620 20:08:33.249204 5868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:08:33.256979 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 5868
I20260620 20:08:33.257102 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:08:33.257367 5486 external_mini_cluster.cc:1428] Setting key ced719b0fda9f5e6f70c2df0979c531b
Jun 20 20:08:33 dist-test-slave-l8h4 krb5kdc[5493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986113, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:33.268183 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:33.251105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:58669 (local address 127.5.91.190:39033)
0620 20:08:33.251253 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:33.251257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:33.252193 (+ 936us) server_negotiation.cc:407] Connection header received
0620 20:08:33.253951 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:33.253955 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:33.254007 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:33.254098 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:33.255777 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:33.256476 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:33.258551 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:33.258881 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:33.259046 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:33.262888 (+ 3842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:33.262893 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:33.262896 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:33.262933 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:33.264744 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:33.265604 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:33.265610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:33.265612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:33.265757 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:33.266447 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:33.266451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:33.266453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:33.266690 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:33.266786 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:33.267897 (+ 1111us) server_negotiation.cc:299] Negotiation successful
0620 20:08:33.268044 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 20:08:33.269089 5999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:33.251455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:39033 (local address 127.5.91.131:58669)
0620 20:08:33.251962 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:33.252009 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:33.253541 (+ 1532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:33.254367 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:33.254377 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:33.254885 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:33.255549 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:33.255568 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:33.256693 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:33.256697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:33.258297 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:33.258318 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:33.259203 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:33.259206 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:33.259412 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:33.260082 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:33.260100 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:33.262365 (+ 2265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:33.264965 (+ 2600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:33.264972 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:33.264989 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:33.265303 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:33.265969 (+ 666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:33.265973 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:33.265975 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:33.266259 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:33.267256 (+ 997us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:33.267264 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:33.267575 (+ 311us) client_negotiation.cc:769] Sending connection context
0620 20:08:33.267756 (+ 181us) client_negotiation.cc:240] Negotiation successful
0620 20:08:33.268105 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":124,"threads_started":1}
I20260620 20:08:33.270776 5997 heartbeater.cc:347] Connected to a master server at 127.5.91.190:39033
I20260620 20:08:33.271037 5997 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:33.271719 5997 heartbeater.cc:511] Master 127.5.91.190:39033 requested a full tablet report, sending...
I20260620 20:08:33.273232 5535 ts_manager.cc:194] Registered new tserver with Master: 1fbcafcc449449c4ab63d00b0ba10de6 (127.5.91.131:41955)
I20260620 20:08:33.273938 5535 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:58669
I20260620 20:08:33.274231 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 20:08:33 dist-test-slave-l8h4 krb5kdc[5493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986113, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 20:08:33 dist-test-slave-l8h4 krb5kdc[5493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986113, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.91.129@KRBTEST.COM
I20260620 20:08:33.302338 6004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:33.291529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46216 (local address 127.5.91.129:39261)
0620 20:08:33.291808 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:33.291816 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:33.291845 (+ 29us) server_negotiation.cc:407] Connection header received
0620 20:08:33.292244 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:33.292250 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:33.292589 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:33.292747 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:33.293470 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:33.294313 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:33.295131 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:33.295497 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:33.295589 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:33.298261 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:33.298270 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:33.298295 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:33.298328 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:33.300299 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:33.300742 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:33.300748 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:33.300749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:33.300831 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:33.301075 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:33.301078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:33.301080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:33.301418 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:33.301724 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:33.301988 (+ 264us) server_negotiation.cc:299] Negotiation successful
0620 20:08:33.302198 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":86,"threads_started":1}
W20260620 20:08:33.303308 5643 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46216
Jun 20 20:08:33 dist-test-slave-l8h4 krb5kdc[5493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 20:08:33 dist-test-slave-l8h4 krb5kdc[5493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.129@KRBTEST.COM
I20260620 20:08:33.332643 6004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:33.323141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46218 (local address 127.5.91.129:39261)
0620 20:08:33.323287 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:33.323291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:33.323479 (+ 188us) server_negotiation.cc:407] Connection header received
0620 20:08:33.323630 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:33.323633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:33.323712 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:33.323868 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:33.324752 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:33.325265 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:33.326201 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:33.326548 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:33.326640 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:33.329193 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:33.329197 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:33.329200 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:33.329243 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:33.330888 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:33.331374 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:33.331379 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:33.331380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:33.331513 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:33.331817 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:33.331820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:33.331821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:33.332038 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:33.332189 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:33.332390 (+ 201us) server_negotiation.cc:299] Negotiation successful
0620 20:08:33.332513 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 20:08:33.333992 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 5600
I20260620 20:08:33.339107 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 5734
I20260620 20:08:33.346520 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 5868
I20260620 20:08:33.351266 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 5506
2026-06-20T20:08:33Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2206 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:08:33 dist-test-slave-l8h4 krb5kdc[6011](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:08:33 dist-test-slave-l8h4 krb5kdc[6011](info): set up 2 sockets
Jun 20 20:08:33 dist-test-slave-l8h4 krb5kdc[6011](info): commencing operation
krb5kdc: starting...
W20260620 20:08:34.378605 5486 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:08:34 dist-test-slave-l8h4 krb5kdc[6011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:08:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:08:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:08:34.509711 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:36571
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:39875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:36571
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:34.623346 6024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:34.623632 6024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:34.623689 6024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:34.628122 6024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:08:34.628199 6024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:34.628230 6024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:34.628252 6024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:08:34.628274 6024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:08:34.634070 6024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:39875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:36571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:36571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:34.635201 6024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:34.636200 6024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:34.644737 6032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:34.644750 6030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:34.644955 6024 server_base.cc:1034] running on GCE node
W20260620 20:08:34.644845 6029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:34.645816 6024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:34.647017 6024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:34.648234 6024 hybrid_clock.cc:648] HybridClock initialized: now 1781986114648176 us; error 97 us; skew 500 ppm
Jun 20 20:08:34 dist-test-slave-l8h4 krb5kdc[6011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986114, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:34.651353 6024 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:08:34.652315 6024 webserver.cc:460] Webserver started at http://127.5.91.190:40859/ using document root <none> and password file <none>
I20260620 20:08:34.652866 6024 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:34.652932 6024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:34.653117 6024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:34.655174 6024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "e94cd2befeab45ecafae41b66071d529"
format_stamp: "Formatted at 2026-06-20 20:08:34 on dist-test-slave-l8h4"
server_key: "5e8ed69efa1e1aa17e6e5e78951c1f3d"
server_key_iv: "b3e310310b8845299813be5ae3282f95"
server_key_version: "encryptionkey@0"
I20260620 20:08:34.655700 6024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "e94cd2befeab45ecafae41b66071d529"
format_stamp: "Formatted at 2026-06-20 20:08:34 on dist-test-slave-l8h4"
server_key: "5e8ed69efa1e1aa17e6e5e78951c1f3d"
server_key_iv: "b3e310310b8845299813be5ae3282f95"
server_key_version: "encryptionkey@0"
I20260620 20:08:34.659471 6024 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260620 20:08:34.662386 6039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:34.664011 6024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:08:34.664149 6024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "e94cd2befeab45ecafae41b66071d529"
format_stamp: "Formatted at 2026-06-20 20:08:34 on dist-test-slave-l8h4"
server_key: "5e8ed69efa1e1aa17e6e5e78951c1f3d"
server_key_iv: "b3e310310b8845299813be5ae3282f95"
server_key_version: "encryptionkey@0"
I20260620 20:08:34.664269 6024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:34.689294 6024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:34.695012 6024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:34.695223 6024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:34.702904 6024 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:36571
I20260620 20:08:34.702931 6091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:36571 every 8 connection(s)
I20260620 20:08:34.704049 6024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:08:34.707021 6092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:34.707381 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 6024
I20260620 20:08:34.707497 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:08:34.707882 5486 external_mini_cluster.cc:1428] Setting key 74a4fcb4d034308b54447452bf363517
I20260620 20:08:34.713678 6092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e94cd2befeab45ecafae41b66071d529: Bootstrap starting.
I20260620 20:08:34.715987 6092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e94cd2befeab45ecafae41b66071d529: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:34.716812 6092 log.cc:824] T 00000000000000000000000000000000 P e94cd2befeab45ecafae41b66071d529: Log is configured to *not* fsync() on all Append() calls
Jun 20 20:08:34 dist-test-slave-l8h4 krb5kdc[6011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:34.719892 6092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e94cd2befeab45ecafae41b66071d529: No bootstrap required, opened a new log
I20260620 20:08:34.724977 6095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:34.709772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56578 (local address 127.5.91.190:36571)
0620 20:08:34.710297 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:34.710312 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:34.710357 (+ 45us) server_negotiation.cc:407] Connection header received
0620 20:08:34.711288 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:34.711325 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:34.711931 (+ 606us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:34.712293 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:34.713064 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:34.714064 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:34.715107 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:34.715668 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:34.715825 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:34.720219 (+ 4394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:34.720225 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:34.720239 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:34.720269 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:34.722011 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:34.722519 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:34.722524 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:34.722529 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:34.722643 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:34.722905 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:34.722909 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:34.722910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:34.723322 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:34.723516 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:34.723797 (+ 281us) server_negotiation.cc:299] Negotiation successful
0620 20:08:34.724190 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":129,"threads_started":1}
I20260620 20:08:34.725505 6092 raft_consensus.cc:348] T 00000000000000000000000000000000 P e94cd2befeab45ecafae41b66071d529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e94cd2befeab45ecafae41b66071d529" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 36571 } }
I20260620 20:08:34.725757 6092 raft_consensus.cc:374] T 00000000000000000000000000000000 P e94cd2befeab45ecafae41b66071d529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:34.725858 6092 raft_consensus.cc:729] T 00000000000000000000000000000000 P e94cd2befeab45ecafae41b66071d529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e94cd2befeab45ecafae41b66071d529, State: Initialized, Role: FOLLOWER
I20260620 20:08:34.726469 6092 consensus_queue.cc:260] T 00000000000000000000000000000000 P e94cd2befeab45ecafae41b66071d529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e94cd2befeab45ecafae41b66071d529" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 36571 } }
I20260620 20:08:34.726758 6092 raft_consensus.cc:388] T 00000000000000000000000000000000 P e94cd2befeab45ecafae41b66071d529 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:08:34.726841 6092 raft_consensus.cc:482] T 00000000000000000000000000000000 P e94cd2befeab45ecafae41b66071d529 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:08:34.726940 6092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e94cd2befeab45ecafae41b66071d529 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:34.728062 6092 raft_consensus.cc:504] T 00000000000000000000000000000000 P e94cd2befeab45ecafae41b66071d529 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e94cd2befeab45ecafae41b66071d529" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 36571 } }
I20260620 20:08:34.728425 6092 leader_election.cc:302] T 00000000000000000000000000000000 P e94cd2befeab45ecafae41b66071d529 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e94cd2befeab45ecafae41b66071d529; no voters:
I20260620 20:08:34.728726 6092 leader_election.cc:288] T 00000000000000000000000000000000 P e94cd2befeab45ecafae41b66071d529 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:08:34.728855 6097 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e94cd2befeab45ecafae41b66071d529 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:34.729077 6097 raft_consensus.cc:686] T 00000000000000000000000000000000 P e94cd2befeab45ecafae41b66071d529 [term 1 LEADER]: Becoming Leader. State: Replica: e94cd2befeab45ecafae41b66071d529, State: Running, Role: LEADER
I20260620 20:08:34.729429 6097 consensus_queue.cc:237] T 00000000000000000000000000000000 P e94cd2befeab45ecafae41b66071d529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e94cd2befeab45ecafae41b66071d529" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 36571 } }
I20260620 20:08:34.729979 6092 sys_catalog.cc:564] T 00000000000000000000000000000000 P e94cd2befeab45ecafae41b66071d529 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:08:34.731184 6099 sys_catalog.cc:455] T 00000000000000000000000000000000 P e94cd2befeab45ecafae41b66071d529 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e94cd2befeab45ecafae41b66071d529. Latest consensus state: current_term: 1 leader_uuid: "e94cd2befeab45ecafae41b66071d529" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e94cd2befeab45ecafae41b66071d529" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 36571 } } }
I20260620 20:08:34.731185 6098 sys_catalog.cc:455] T 00000000000000000000000000000000 P e94cd2befeab45ecafae41b66071d529 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e94cd2befeab45ecafae41b66071d529" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e94cd2befeab45ecafae41b66071d529" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 36571 } } }
I20260620 20:08:34.731343 6099 sys_catalog.cc:458] T 00000000000000000000000000000000 P e94cd2befeab45ecafae41b66071d529 [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:34.731341 6098 sys_catalog.cc:458] T 00000000000000000000000000000000 P e94cd2befeab45ecafae41b66071d529 [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:34.731725 6106 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:08:34.736778 6106 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:08:34.743924 6106 catalog_manager.cc:1347] Generated new cluster ID: 0d3e0e08987d4103aaaf6e17ffea508e
I20260620 20:08:34.744019 6106 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:08:34.757372 6106 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:08:34.758829 6106 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:08:34.770663 6106 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e94cd2befeab45ecafae41b66071d529: Generated new TSK 0
I20260620 20:08:34.771564 6106 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.91.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
I20260620 20:08:34.806931 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:36571
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:39875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:34.926086 6118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:34.926374 6118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:34.926419 6118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:34.931444 6118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:34.931519 6118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:34.931589 6118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:08:34.939138 6118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:39875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:36571
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:34.940188 6118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:34.941174 6118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:34.951587 6124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:34.951678 6123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:34.952092 6126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:34.952235 6118 server_base.cc:1034] running on GCE node
I20260620 20:08:34.952677 6118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:34.953361 6118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:34.954560 6118 hybrid_clock.cc:648] HybridClock initialized: now 1781986114954518 us; error 56 us; skew 500 ppm
Jun 20 20:08:34 dist-test-slave-l8h4 krb5kdc[6011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986114, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:34.957617 6118 init.cc:377] Logged in from keytab as kudu/127.5.91.129@KRBTEST.COM (short username kudu)
I20260620 20:08:34.958513 6118 webserver.cc:460] Webserver started at http://127.5.91.129:34561/ using document root <none> and password file <none>
I20260620 20:08:34.959064 6118 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:34.959128 6118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:34.959484 6118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:34.961622 6118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "fffa3f343b2440adbe6eda6854fa80b1"
format_stamp: "Formatted at 2026-06-20 20:08:34 on dist-test-slave-l8h4"
server_key: "0881e267ac40a6cd4a982863bb5e2ebf"
server_key_iv: "c18589df3616b9455e28b2dc57d3e6d7"
server_key_version: "encryptionkey@0"
I20260620 20:08:34.962203 6118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "fffa3f343b2440adbe6eda6854fa80b1"
format_stamp: "Formatted at 2026-06-20 20:08:34 on dist-test-slave-l8h4"
server_key: "0881e267ac40a6cd4a982863bb5e2ebf"
server_key_iv: "c18589df3616b9455e28b2dc57d3e6d7"
server_key_version: "encryptionkey@0"
I20260620 20:08:34.965857 6118 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:08:34.968640 6133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:34.970031 6118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 20:08:34.970170 6118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "fffa3f343b2440adbe6eda6854fa80b1"
format_stamp: "Formatted at 2026-06-20 20:08:34 on dist-test-slave-l8h4"
server_key: "0881e267ac40a6cd4a982863bb5e2ebf"
server_key_iv: "c18589df3616b9455e28b2dc57d3e6d7"
server_key_version: "encryptionkey@0"
I20260620 20:08:34.970288 6118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:34.993211 6118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:34.996567 6118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:34.996730 6118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:34.997277 6118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:08:34.998875 6118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:34.998939 6118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:34.998984 6118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:34.998999 6118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:35.008450 6118 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:44547
I20260620 20:08:35.008471 6246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:44547 every 8 connection(s)
I20260620 20:08:35.009400 6118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:08:35.014365 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 6118
I20260620 20:08:35.014510 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:08:35.014768 5486 external_mini_cluster.cc:1428] Setting key 22abc84d866a8ce760b2024991740495
Jun 20 20:08:35 dist-test-slave-l8h4 krb5kdc[6011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986114, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.91.130@KRBTEST.COM; defaulting to no policy
I20260620 20:08:35.026830 6095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:35.012088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:56107 (local address 127.5.91.190:36571)
0620 20:08:35.012251 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:35.012254 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:35.013064 (+ 810us) server_negotiation.cc:407] Connection header received
0620 20:08:35.015325 (+ 2261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:35.015328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:35.015392 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:35.015470 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:35.016863 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.017457 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.018781 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.019109 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.019213 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:35.022111 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:35.022116 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:35.022119 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:35.022177 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:35.023800 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:35.024726 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:35.024731 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:35.024733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:35.024892 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:35.025347 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:35.025351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:35.025352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:35.025730 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:35.025833 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:35.026560 (+ 727us) server_negotiation.cc:299] Negotiation successful
0620 20:08:35.026697 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.130@KRBTEST.COM" created.
I20260620 20:08:35.027705 6249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:35.012340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:36571 (local address 127.5.91.129:56107)
0620 20:08:35.012852 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:35.012896 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:35.015067 (+ 2171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:35.015666 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:35.015673 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:35.016046 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:35.016693 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:35.016714 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.017656 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.017660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:35.018660 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:35.018670 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.019275 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.019278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:35.019447 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:35.020095 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:35.020202 (+ 107us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:35.021898 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:35.024089 (+ 2191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:35.024101 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:35.024119 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:35.024403 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:35.025076 (+ 673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:35.025079 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:35.025080 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:35.025231 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:35.025866 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:35.025872 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:35.026128 (+ 256us) client_negotiation.cc:769] Sending connection context
0620 20:08:35.026438 (+ 310us) client_negotiation.cc:240] Negotiation successful
0620 20:08:35.026739 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":147,"threads_started":1}
I20260620 20:08:35.028983 6247 heartbeater.cc:347] Connected to a master server at 127.5.91.190:36571
I20260620 20:08:35.029218 6247 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:35.029757 6247 heartbeater.cc:511] Master 127.5.91.190:36571 requested a full tablet report, sending...
I20260620 20:08:35.031677 6056 ts_manager.cc:194] Registered new tserver with Master: fffa3f343b2440adbe6eda6854fa80b1 (127.5.91.129:44547)
I20260620 20:08:35.032927 6056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:56107
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
I20260620 20:08:35.046816 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:36571
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:39875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:35.165318 6252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:35.165591 6252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:35.165633 6252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:35.169745 6252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:35.169816 6252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:35.169885 6252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:08:35.175328 6252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:39875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:36571
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:35.176427 6252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:35.177418 6252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:35.187204 6258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:35.187299 6257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:35.187407 6252 server_base.cc:1034] running on GCE node
W20260620 20:08:35.187301 6260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:35.187909 6252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:35.188544 6252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:35.189761 6252 hybrid_clock.cc:648] HybridClock initialized: now 1781986115189716 us; error 53 us; skew 500 ppm
Jun 20 20:08:35 dist-test-slave-l8h4 krb5kdc[6011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986115, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:35.192574 6252 init.cc:377] Logged in from keytab as kudu/127.5.91.130@KRBTEST.COM (short username kudu)
I20260620 20:08:35.193506 6252 webserver.cc:460] Webserver started at http://127.5.91.130:43393/ using document root <none> and password file <none>
I20260620 20:08:35.194093 6252 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:35.194200 6252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:35.194402 6252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:35.196482 6252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "1312405dc8264303840abfe892b162a3"
format_stamp: "Formatted at 2026-06-20 20:08:35 on dist-test-slave-l8h4"
server_key: "6d9c36f1b4d322314896f57e89f37127"
server_key_iv: "53cee23c4480b0d08715dc7a13ba9249"
server_key_version: "encryptionkey@0"
I20260620 20:08:35.197027 6252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "1312405dc8264303840abfe892b162a3"
format_stamp: "Formatted at 2026-06-20 20:08:35 on dist-test-slave-l8h4"
server_key: "6d9c36f1b4d322314896f57e89f37127"
server_key_iv: "53cee23c4480b0d08715dc7a13ba9249"
server_key_version: "encryptionkey@0"
I20260620 20:08:35.200785 6252 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260620 20:08:35.203807 6267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:35.205602 6252 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 20:08:35.205731 6252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "1312405dc8264303840abfe892b162a3"
format_stamp: "Formatted at 2026-06-20 20:08:35 on dist-test-slave-l8h4"
server_key: "6d9c36f1b4d322314896f57e89f37127"
server_key_iv: "53cee23c4480b0d08715dc7a13ba9249"
server_key_version: "encryptionkey@0"
I20260620 20:08:35.205891 6252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:35.220026 6252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:35.223671 6252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:35.223867 6252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:35.224457 6252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:08:35.225281 6252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:35.225328 6252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:35.225380 6252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:35.225409 6252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:35.235396 6252 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:42893
I20260620 20:08:35.235422 6380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:42893 every 8 connection(s)
I20260620 20:08:35.236250 6252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:08:35.244582 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 6252
I20260620 20:08:35.244702 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:08:35.244930 5486 external_mini_cluster.cc:1428] Setting key 47b61cdb9ef9081b62bcdf54a3d95b0d
Jun 20 20:08:35 dist-test-slave-l8h4 krb5kdc[6011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986115, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:35.249521 6095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:35.237993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:34059 (local address 127.5.91.190:36571)
0620 20:08:35.238182 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:35.238188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:35.239043 (+ 855us) server_negotiation.cc:407] Connection header received
0620 20:08:35.240351 (+ 1308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:35.240354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:35.240411 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:35.240502 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:35.241631 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.242173 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.243272 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.243498 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.243586 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:35.246284 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:35.246288 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:35.246291 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:35.246317 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:35.247655 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:35.248132 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:35.248136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:35.248137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:35.248208 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:35.248507 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:35.248510 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:35.248511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:35.248679 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:35.248760 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:35.249238 (+ 478us) server_negotiation.cc:299] Negotiation successful
0620 20:08:35.249364 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 20:08:35.250401 6383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:35.238432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:36571 (local address 127.5.91.130:34059)
0620 20:08:35.238902 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:35.238943 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:35.240144 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:35.240666 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:35.240673 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:35.241018 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:35.241499 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:35.241513 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.242277 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.242280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:35.243155 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:35.243169 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.243638 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.243643 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:35.243805 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:35.244504 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:35.244521 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:35.246025 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:35.247764 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:35.247771 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:35.247787 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:35.248041 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:35.248288 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:35.248291 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:35.248293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:35.248422 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:35.248771 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:35.248778 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:35.249023 (+ 245us) client_negotiation.cc:769] Sending connection context
0620 20:08:35.249186 (+ 163us) client_negotiation.cc:240] Negotiation successful
0620 20:08:35.249502 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":114,"threads_started":1}
I20260620 20:08:35.251487 6381 heartbeater.cc:347] Connected to a master server at 127.5.91.190:36571
I20260620 20:08:35.251760 6381 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:35.252364 6381 heartbeater.cc:511] Master 127.5.91.190:36571 requested a full tablet report, sending...
I20260620 20:08:35.253604 6056 ts_manager.cc:194] Registered new tserver with Master: 1312405dc8264303840abfe892b162a3 (127.5.91.130:42893)
WARNING: no policy specified for kudu/127.5.91.131@KRBTEST.COM; defaulting to no policy
I20260620 20:08:35.254421 6056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:34059
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
I20260620 20:08:35.278050 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:36571
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:39875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:35.402526 6386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:35.402792 6386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:35.402851 6386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:35.407368 6386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:35.407560 6386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:35.407763 6386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:08:35.413551 6386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:39875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:36571
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:35.414710 6386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:35.415694 6386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:35.426081 6392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:35.426086 6391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:35.426086 6394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:35.426445 6386 server_base.cc:1034] running on GCE node
I20260620 20:08:35.426838 6386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:35.427456 6386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:35.428633 6386 hybrid_clock.cc:648] HybridClock initialized: now 1781986115428614 us; error 30 us; skew 500 ppm
Jun 20 20:08:35 dist-test-slave-l8h4 krb5kdc[6011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986115, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:35.431856 6386 init.cc:377] Logged in from keytab as kudu/127.5.91.131@KRBTEST.COM (short username kudu)
I20260620 20:08:35.432705 6386 webserver.cc:460] Webserver started at http://127.5.91.131:40985/ using document root <none> and password file <none>
I20260620 20:08:35.433224 6386 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:35.433287 6386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:35.433463 6386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:35.435427 6386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "9947282e6b4344eba97a20d3232e0ec3"
format_stamp: "Formatted at 2026-06-20 20:08:35 on dist-test-slave-l8h4"
server_key: "8f5b72877e53d4a041d71b6041c27aec"
server_key_iv: "08ecdc0c7fb175212c9172428dbd3294"
server_key_version: "encryptionkey@0"
I20260620 20:08:35.435940 6386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "9947282e6b4344eba97a20d3232e0ec3"
format_stamp: "Formatted at 2026-06-20 20:08:35 on dist-test-slave-l8h4"
server_key: "8f5b72877e53d4a041d71b6041c27aec"
server_key_iv: "08ecdc0c7fb175212c9172428dbd3294"
server_key_version: "encryptionkey@0"
I20260620 20:08:35.439486 6386 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 20:08:35.441957 6401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:35.443261 6386 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:08:35.443404 6386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "9947282e6b4344eba97a20d3232e0ec3"
format_stamp: "Formatted at 2026-06-20 20:08:35 on dist-test-slave-l8h4"
server_key: "8f5b72877e53d4a041d71b6041c27aec"
server_key_iv: "08ecdc0c7fb175212c9172428dbd3294"
server_key_version: "encryptionkey@0"
I20260620 20:08:35.443497 6386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:35.457597 6386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:35.461310 6386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:35.461494 6386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:35.462054 6386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:08:35.462903 6386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:35.462949 6386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:35.463001 6386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:35.463027 6386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:35.471561 6386 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:37947
I20260620 20:08:35.471634 6514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:37947 every 8 connection(s)
I20260620 20:08:35.472442 6386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:08:35.475354 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 6386
I20260620 20:08:35.475474 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:08:35.475759 5486 external_mini_cluster.cc:1428] Setting key a57158ad5479fe8a6bfd314a6be850c6
Jun 20 20:08:35 dist-test-slave-l8h4 krb5kdc[6011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986115, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:35.486656 6095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:35.474245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:41023 (local address 127.5.91.190:36571)
0620 20:08:35.474400 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:35.474404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:35.475552 (+ 1148us) server_negotiation.cc:407] Connection header received
0620 20:08:35.476996 (+ 1444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:35.476999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:35.477057 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:35.477134 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:35.478311 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.478845 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.479941 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.480159 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.480244 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:35.483247 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:35.483251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:35.483252 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:35.483275 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:35.484589 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:35.485156 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:35.485159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:35.485160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:35.485230 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:35.485544 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:35.485547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:35.485548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:35.485717 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:35.485797 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:35.486344 (+ 547us) server_negotiation.cc:299] Negotiation successful
0620 20:08:35.486490 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 20:08:35.487442 6517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:35.474761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:36571 (local address 127.5.91.131:41023)
0620 20:08:35.475373 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:35.475427 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:35.476776 (+ 1349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:35.477328 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:35.477337 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:35.477676 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:35.478186 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:35.478204 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.478959 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.478963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:35.479822 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:35.479830 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.480287 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.480291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:35.480470 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:35.481209 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:35.481233 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:35.483078 (+ 1845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:35.484757 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:35.484762 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:35.484774 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:35.485045 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:35.485311 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:35.485314 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:35.485316 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:35.485450 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:35.485810 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:35.485816 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:35.486075 (+ 259us) client_negotiation.cc:769] Sending connection context
0620 20:08:35.486277 (+ 202us) client_negotiation.cc:240] Negotiation successful
0620 20:08:35.486629 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":161,"threads_started":1}
I20260620 20:08:35.488613 6515 heartbeater.cc:347] Connected to a master server at 127.5.91.190:36571
I20260620 20:08:35.488831 6515 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:35.489287 6515 heartbeater.cc:511] Master 127.5.91.190:36571 requested a full tablet report, sending...
I20260620 20:08:35.490375 6056 ts_manager.cc:194] Registered new tserver with Master: 9947282e6b4344eba97a20d3232e0ec3 (127.5.91.131:37947)
I20260620 20:08:35.490937 6056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:41023
I20260620 20:08:35.502908 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 20:08:35.514966 6095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:35.507157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56582 (local address 127.5.91.190:36571)
0620 20:08:35.507326 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:35.507330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:35.507495 (+ 165us) server_negotiation.cc:407] Connection header received
0620 20:08:35.507625 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:35.507628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:35.507673 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:35.507738 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:35.508409 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.508986 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.509854 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.510071 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.510184 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:35.511597 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:35.511601 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:35.511604 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:35.511629 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:35.513206 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:35.513644 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:35.513648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:35.513650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:35.513775 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:35.514073 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:35.514076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:35.514078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:35.514344 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:35.514468 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:35.514697 (+ 229us) server_negotiation.cc:299] Negotiation successful
0620 20:08:35.514814 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260620 20:08:35.521844 6056 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56582:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:08:35.525754 6056 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:08:35.541702 6526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:35.536144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:37947 (local address 127.0.0.1:47236)
0620 20:08:35.536401 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:35.536413 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:35.536471 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:35.537191 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:35.537197 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:35.537233 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:35.537744 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:35.537752 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.539133 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.539136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:35.540262 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:35.540271 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.541174 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.541177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:35.541284 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:35.541320 (+ 36us) client_negotiation.cc:769] Sending connection context
0620 20:08:35.541426 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 20:08:35.541541 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":155,"spinlock_wait_cycles":6016,"thread_start_us":77,"threads_started":1}
I20260620 20:08:35.541709 6531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:35.536400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47236 (local address 127.5.91.131:37947)
0620 20:08:35.536771 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:35.536784 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:35.536805 (+ 21us) server_negotiation.cc:407] Connection header received
0620 20:08:35.536885 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:35.536890 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:35.537065 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:35.537202 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:35.537901 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.539007 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.540374 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.541057 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.541162 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:35.541306 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:35.541462 (+ 156us) server_negotiation.cc:299] Negotiation successful
0620 20:08:35.541556 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":86,"threads_started":1}
I20260620 20:08:35.541901 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:35.535514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52942 (local address 127.5.91.130:42893)
0620 20:08:35.536060 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:35.536073 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:35.536366 (+ 293us) server_negotiation.cc:407] Connection header received
0620 20:08:35.536633 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:35.536638 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:35.536843 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:35.536970 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:35.537932 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.538715 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.540472 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.541141 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.541215 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:35.541351 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:35.541656 (+ 305us) server_negotiation.cc:299] Negotiation successful
0620 20:08:35.541759 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":242,"threads_started":1}
I20260620 20:08:35.542336 6527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:35.535389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:42893 (local address 127.0.0.1:52942)
0620 20:08:35.536235 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:35.536283 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:35.536469 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:35.537191 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:35.537197 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:35.537231 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:35.537719 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:35.537732 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.538836 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.538839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:35.540313 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:35.540323 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.541440 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.541443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:35.541520 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:35.541538 (+ 18us) client_negotiation.cc:769] Sending connection context
0620 20:08:35.542069 (+ 531us) client_negotiation.cc:240] Negotiation successful
0620 20:08:35.542198 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":250,"spinlock_wait_cycles":5760,"thread_start_us":60,"threads_started":1}
I20260620 20:08:35.542369 6529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:35.535809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55338 (local address 127.5.91.129:44547)
0620 20:08:35.536135 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:35.536146 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:35.536838 (+ 692us) server_negotiation.cc:407] Connection header received
0620 20:08:35.537051 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:35.537056 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:35.537217 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:35.537315 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:35.537705 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.538919 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.540773 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.541528 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.541624 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:35.541748 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:35.542101 (+ 353us) server_negotiation.cc:299] Negotiation successful
0620 20:08:35.542232 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":96,"threads_started":1}
I20260620 20:08:35.544118 6530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:35.535475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:44547 (local address 127.0.0.1:55338)
0620 20:08:35.536749 (+ 1274us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:35.536761 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:35.536903 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:35.537458 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:35.537460 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:35.537474 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:35.537593 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:35.537597 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.539025 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.539028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:35.540649 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:35.540658 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.541875 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.541879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:35.541973 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:35.541998 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 20:08:35.542070 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 20:08:35.542180 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"mutex_wait_us":59,"thread_start_us":115,"threads_started":1}
I20260620 20:08:35.545485 6449 tablet_service.cc:1467] Processing CreateTablet for tablet 8118f7f64ab2437f99323e5f446d71c2 (DEFAULT_TABLE table=test-table [id=ee5fd0a53c6c4530818654be4946eed7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:35.545605 6315 tablet_service.cc:1467] Processing CreateTablet for tablet 8118f7f64ab2437f99323e5f446d71c2 (DEFAULT_TABLE table=test-table [id=ee5fd0a53c6c4530818654be4946eed7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:35.546463 6449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8118f7f64ab2437f99323e5f446d71c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:35.546487 6315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8118f7f64ab2437f99323e5f446d71c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:35.552484 6533 tablet_bootstrap.cc:492] T 8118f7f64ab2437f99323e5f446d71c2 P 1312405dc8264303840abfe892b162a3: Bootstrap starting.
I20260620 20:08:35.554479 6532 tablet_bootstrap.cc:492] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3: Bootstrap starting.
I20260620 20:08:35.555294 6533 tablet_bootstrap.cc:654] T 8118f7f64ab2437f99323e5f446d71c2 P 1312405dc8264303840abfe892b162a3: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:35.555053 6181 tablet_service.cc:1467] Processing CreateTablet for tablet 8118f7f64ab2437f99323e5f446d71c2 (DEFAULT_TABLE table=test-table [id=ee5fd0a53c6c4530818654be4946eed7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:35.555922 6181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8118f7f64ab2437f99323e5f446d71c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:35.556190 6533 log.cc:824] T 8118f7f64ab2437f99323e5f446d71c2 P 1312405dc8264303840abfe892b162a3: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:35.556559 6532 tablet_bootstrap.cc:654] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:35.557657 6532 log.cc:824] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:35.558545 6533 tablet_bootstrap.cc:492] T 8118f7f64ab2437f99323e5f446d71c2 P 1312405dc8264303840abfe892b162a3: No bootstrap required, opened a new log
I20260620 20:08:35.558736 6533 ts_tablet_manager.cc:1397] T 8118f7f64ab2437f99323e5f446d71c2 P 1312405dc8264303840abfe892b162a3: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260620 20:08:35.559655 6532 tablet_bootstrap.cc:492] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3: No bootstrap required, opened a new log
I20260620 20:08:35.559842 6532 ts_tablet_manager.cc:1397] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 20:08:35.561937 6536 tablet_bootstrap.cc:492] T 8118f7f64ab2437f99323e5f446d71c2 P fffa3f343b2440adbe6eda6854fa80b1: Bootstrap starting.
I20260620 20:08:35.563021 6533 raft_consensus.cc:348] T 8118f7f64ab2437f99323e5f446d71c2 P 1312405dc8264303840abfe892b162a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1312405dc8264303840abfe892b162a3" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 42893 } } peers { permanent_uuid: "fffa3f343b2440adbe6eda6854fa80b1" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44547 } } peers { permanent_uuid: "9947282e6b4344eba97a20d3232e0ec3" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37947 } }
I20260620 20:08:35.563299 6533 raft_consensus.cc:374] T 8118f7f64ab2437f99323e5f446d71c2 P 1312405dc8264303840abfe892b162a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:35.563364 6533 raft_consensus.cc:729] T 8118f7f64ab2437f99323e5f446d71c2 P 1312405dc8264303840abfe892b162a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1312405dc8264303840abfe892b162a3, State: Initialized, Role: FOLLOWER
I20260620 20:08:35.563416 6532 raft_consensus.cc:348] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1312405dc8264303840abfe892b162a3" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 42893 } } peers { permanent_uuid: "fffa3f343b2440adbe6eda6854fa80b1" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44547 } } peers { permanent_uuid: "9947282e6b4344eba97a20d3232e0ec3" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37947 } }
I20260620 20:08:35.563648 6532 raft_consensus.cc:374] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:35.563707 6532 raft_consensus.cc:729] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9947282e6b4344eba97a20d3232e0ec3, State: Initialized, Role: FOLLOWER
I20260620 20:08:35.563814 6533 consensus_queue.cc:260] T 8118f7f64ab2437f99323e5f446d71c2 P 1312405dc8264303840abfe892b162a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1312405dc8264303840abfe892b162a3" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 42893 } } peers { permanent_uuid: "fffa3f343b2440adbe6eda6854fa80b1" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44547 } } peers { permanent_uuid: "9947282e6b4344eba97a20d3232e0ec3" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37947 } }
I20260620 20:08:35.563946 6536 tablet_bootstrap.cc:654] T 8118f7f64ab2437f99323e5f446d71c2 P fffa3f343b2440adbe6eda6854fa80b1: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:35.564112 6532 consensus_queue.cc:260] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1312405dc8264303840abfe892b162a3" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 42893 } } peers { permanent_uuid: "fffa3f343b2440adbe6eda6854fa80b1" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44547 } } peers { permanent_uuid: "9947282e6b4344eba97a20d3232e0ec3" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37947 } }
I20260620 20:08:35.564725 6536 log.cc:824] T 8118f7f64ab2437f99323e5f446d71c2 P fffa3f343b2440adbe6eda6854fa80b1: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:35.564994 6533 ts_tablet_manager.cc:1428] T 8118f7f64ab2437f99323e5f446d71c2 P 1312405dc8264303840abfe892b162a3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260620 20:08:35.564994 6532 ts_tablet_manager.cc:1428] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260620 20:08:35.565392 6381 heartbeater.cc:503] Master 127.5.91.190:36571 was elected leader, sending a full tablet report...
I20260620 20:08:35.565785 6515 heartbeater.cc:503] Master 127.5.91.190:36571 was elected leader, sending a full tablet report...
I20260620 20:08:35.566848 6536 tablet_bootstrap.cc:492] T 8118f7f64ab2437f99323e5f446d71c2 P fffa3f343b2440adbe6eda6854fa80b1: No bootstrap required, opened a new log
I20260620 20:08:35.567015 6536 ts_tablet_manager.cc:1397] T 8118f7f64ab2437f99323e5f446d71c2 P fffa3f343b2440adbe6eda6854fa80b1: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260620 20:08:35.570641 6536 raft_consensus.cc:348] T 8118f7f64ab2437f99323e5f446d71c2 P fffa3f343b2440adbe6eda6854fa80b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1312405dc8264303840abfe892b162a3" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 42893 } } peers { permanent_uuid: "fffa3f343b2440adbe6eda6854fa80b1" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44547 } } peers { permanent_uuid: "9947282e6b4344eba97a20d3232e0ec3" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37947 } }
I20260620 20:08:35.570860 6536 raft_consensus.cc:374] T 8118f7f64ab2437f99323e5f446d71c2 P fffa3f343b2440adbe6eda6854fa80b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:35.570907 6536 raft_consensus.cc:729] T 8118f7f64ab2437f99323e5f446d71c2 P fffa3f343b2440adbe6eda6854fa80b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fffa3f343b2440adbe6eda6854fa80b1, State: Initialized, Role: FOLLOWER
I20260620 20:08:35.571292 6536 consensus_queue.cc:260] T 8118f7f64ab2437f99323e5f446d71c2 P fffa3f343b2440adbe6eda6854fa80b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1312405dc8264303840abfe892b162a3" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 42893 } } peers { permanent_uuid: "fffa3f343b2440adbe6eda6854fa80b1" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44547 } } peers { permanent_uuid: "9947282e6b4344eba97a20d3232e0ec3" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37947 } }
I20260620 20:08:35.572170 6536 ts_tablet_manager.cc:1428] T 8118f7f64ab2437f99323e5f446d71c2 P fffa3f343b2440adbe6eda6854fa80b1: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260620 20:08:35.572345 6247 heartbeater.cc:503] Master 127.5.91.190:36571 was elected leader, sending a full tablet report...
I20260620 20:08:35.614704 6538 raft_consensus.cc:482] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:08:35.614925 6538 raft_consensus.cc:504] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1312405dc8264303840abfe892b162a3" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 42893 } } peers { permanent_uuid: "fffa3f343b2440adbe6eda6854fa80b1" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44547 } } peers { permanent_uuid: "9947282e6b4344eba97a20d3232e0ec3" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37947 } }
I20260620 20:08:35.616318 6538 leader_election.cc:288] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1312405dc8264303840abfe892b162a3 (127.5.91.130:42893), fffa3f343b2440adbe6eda6854fa80b1 (127.5.91.129:44547)
I20260620 20:08:35.620561 6529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:35.616930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:36183 (local address 127.5.91.129:44547)
0620 20:08:35.617051 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:35.617055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:35.617304 (+ 249us) server_negotiation.cc:407] Connection header received
0620 20:08:35.617422 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:35.617425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:35.617468 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:35.617526 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:35.617823 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.618367 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.619563 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.620017 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.620128 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:35.620186 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:35.620354 (+ 168us) server_negotiation.cc:299] Negotiation successful
0620 20:08:35.620441 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 20:08:35.620592 6541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:35.616789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:44547 (local address 127.5.91.131:36183)
0620 20:08:35.617217 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:35.617228 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:35.617321 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:35.617554 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:35.617557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:35.617602 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:35.617716 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:35.617722 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.618470 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.618474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:35.619439 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:35.619449 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.620133 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.620136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:35.620241 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:35.620263 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 20:08:35.620328 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 20:08:35.620403 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":84,"spinlock_wait_cycles":39296,"thread_start_us":192,"threads_started":1}
I20260620 20:08:35.620939 6517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:35.616523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:42893 (local address 127.5.91.131:37319)
0620 20:08:35.616652 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:35.616666 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:35.616757 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:35.617439 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:35.617443 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:35.617615 (+ 172us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:35.617730 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:35.617737 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.618570 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.618574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:35.619660 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:35.619680 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.620504 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.620508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:35.620608 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:35.620629 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 20:08:35.620702 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 20:08:35.620789 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":3,"spinlock_wait_cycles":6016}
I20260620 20:08:35.621186 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:35.616750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:37319 (local address 127.5.91.130:42893)
0620 20:08:35.616928 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:35.616934 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:35.616955 (+ 21us) server_negotiation.cc:407] Connection header received
0620 20:08:35.617100 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:35.617192 (+ 92us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:35.617298 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:35.617412 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:35.617886 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.618462 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.619882 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.620378 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.620511 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:35.620562 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:35.620990 (+ 428us) server_negotiation.cc:299] Negotiation successful
0620 20:08:35.621074 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260620 20:08:35.621656 6201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8118f7f64ab2437f99323e5f446d71c2" candidate_uuid: "9947282e6b4344eba97a20d3232e0ec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fffa3f343b2440adbe6eda6854fa80b1" is_pre_election: true
I20260620 20:08:35.622008 6201 raft_consensus.cc:2454] T 8118f7f64ab2437f99323e5f446d71c2 P fffa3f343b2440adbe6eda6854fa80b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9947282e6b4344eba97a20d3232e0ec3 in term 0.
I20260620 20:08:35.621971 6335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8118f7f64ab2437f99323e5f446d71c2" candidate_uuid: "9947282e6b4344eba97a20d3232e0ec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1312405dc8264303840abfe892b162a3" is_pre_election: true
I20260620 20:08:35.622277 6335 raft_consensus.cc:2454] T 8118f7f64ab2437f99323e5f446d71c2 P 1312405dc8264303840abfe892b162a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9947282e6b4344eba97a20d3232e0ec3 in term 0.
I20260620 20:08:35.622543 6402 leader_election.cc:302] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9947282e6b4344eba97a20d3232e0ec3, fffa3f343b2440adbe6eda6854fa80b1; no voters:
I20260620 20:08:35.622826 6538 raft_consensus.cc:2781] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:08:35.622890 6538 raft_consensus.cc:482] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:08:35.622916 6538 raft_consensus.cc:3037] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:35.623939 6538 raft_consensus.cc:504] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1312405dc8264303840abfe892b162a3" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 42893 } } peers { permanent_uuid: "fffa3f343b2440adbe6eda6854fa80b1" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44547 } } peers { permanent_uuid: "9947282e6b4344eba97a20d3232e0ec3" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37947 } }
I20260620 20:08:35.624306 6538 leader_election.cc:288] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3 [CANDIDATE]: Term 1 election: Requested vote from peers 1312405dc8264303840abfe892b162a3 (127.5.91.130:42893), fffa3f343b2440adbe6eda6854fa80b1 (127.5.91.129:44547)
I20260620 20:08:35.624825 6201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8118f7f64ab2437f99323e5f446d71c2" candidate_uuid: "9947282e6b4344eba97a20d3232e0ec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fffa3f343b2440adbe6eda6854fa80b1"
I20260620 20:08:35.624961 6201 raft_consensus.cc:3037] T 8118f7f64ab2437f99323e5f446d71c2 P fffa3f343b2440adbe6eda6854fa80b1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:35.624960 6335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8118f7f64ab2437f99323e5f446d71c2" candidate_uuid: "9947282e6b4344eba97a20d3232e0ec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1312405dc8264303840abfe892b162a3"
I20260620 20:08:35.625371 6335 raft_consensus.cc:3037] T 8118f7f64ab2437f99323e5f446d71c2 P 1312405dc8264303840abfe892b162a3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:35.626639 6201 raft_consensus.cc:2454] T 8118f7f64ab2437f99323e5f446d71c2 P fffa3f343b2440adbe6eda6854fa80b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9947282e6b4344eba97a20d3232e0ec3 in term 1.
I20260620 20:08:35.627115 6402 leader_election.cc:302] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9947282e6b4344eba97a20d3232e0ec3, fffa3f343b2440adbe6eda6854fa80b1; no voters:
I20260620 20:08:35.627353 6335 raft_consensus.cc:2454] T 8118f7f64ab2437f99323e5f446d71c2 P 1312405dc8264303840abfe892b162a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9947282e6b4344eba97a20d3232e0ec3 in term 1.
I20260620 20:08:35.627393 6538 raft_consensus.cc:2781] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:35.627724 6538 raft_consensus.cc:686] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3 [term 1 LEADER]: Becoming Leader. State: Replica: 9947282e6b4344eba97a20d3232e0ec3, State: Running, Role: LEADER
I20260620 20:08:35.628237 6538 consensus_queue.cc:237] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1312405dc8264303840abfe892b162a3" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 42893 } } peers { permanent_uuid: "fffa3f343b2440adbe6eda6854fa80b1" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44547 } } peers { permanent_uuid: "9947282e6b4344eba97a20d3232e0ec3" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37947 } }
I20260620 20:08:35.632203 6056 catalog_manager.cc:5495] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9947282e6b4344eba97a20d3232e0ec3 (127.5.91.131). New cstate: current_term: 1 leader_uuid: "9947282e6b4344eba97a20d3232e0ec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1312405dc8264303840abfe892b162a3" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 42893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fffa3f343b2440adbe6eda6854fa80b1" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9947282e6b4344eba97a20d3232e0ec3" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37947 } health_report { overall_health: HEALTHY } } }
Jun 20 20:08:35 dist-test-slave-l8h4 krb5kdc[6011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.129@KRBTEST.COM
I20260620 20:08:35.672564 6529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:35.659589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55354 (local address 127.5.91.129:44547)
0620 20:08:35.659729 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:35.659733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:35.659773 (+ 40us) server_negotiation.cc:407] Connection header received
0620 20:08:35.659959 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:35.659961 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:35.660007 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:35.660082 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:35.660807 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.661335 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.662202 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.662426 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.662513 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:35.668937 (+ 6424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:35.668949 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:35.668965 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:35.668994 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:35.670831 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:35.671261 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:35.671266 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:35.671268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:35.671366 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:35.671636 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:35.671639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:35.671640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:35.671884 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:35.672090 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:35.672281 (+ 191us) server_negotiation.cc:299] Negotiation successful
0620 20:08:35.672411 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Jun 20 20:08:35 dist-test-slave-l8h4 krb5kdc[6011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986115, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 20:08:35 dist-test-slave-l8h4 krb5kdc[6011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986115, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.91.129@KRBTEST.COM
I20260620 20:08:35.699045 6529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:35.690963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55356 (local address 127.5.91.129:44547)
0620 20:08:35.691121 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:35.691125 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:35.691331 (+ 206us) server_negotiation.cc:407] Connection header received
0620 20:08:35.691477 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:35.691479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:35.691522 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:35.691587 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:35.692265 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.692768 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.693552 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.693764 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.693847 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:35.696191 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:35.696194 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:35.696196 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:35.696219 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:35.697483 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:35.697896 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:35.697899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:35.697900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:35.697976 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:35.698269 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:35.698272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:35.698273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:35.698469 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:35.698597 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:35.698825 (+ 228us) server_negotiation.cc:299] Negotiation successful
0620 20:08:35.698924 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Jun 20 20:08:35 dist-test-slave-l8h4 krb5kdc[6011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260620 20:08:35.723547 6516 tablet.cc:2369] T 8118f7f64ab2437f99323e5f446d71c2 P 9947282e6b4344eba97a20d3232e0ec3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 20 20:08:35 dist-test-slave-l8h4 krb5kdc[6011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.129@KRBTEST.COM
I20260620 20:08:35.729013 6529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:35.718909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55372 (local address 127.5.91.129:44547)
0620 20:08:35.719096 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:35.719101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:35.719323 (+ 222us) server_negotiation.cc:407] Connection header received
0620 20:08:35.719452 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:35.719454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:35.719514 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:35.719609 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:35.720448 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.720968 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.721854 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:35.722068 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:35.722187 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:35.725021 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:35.725025 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:35.725027 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:35.725051 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:35.726685 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:35.727193 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:35.727198 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:35.727199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:35.727286 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:35.727872 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:35.727876 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:35.727878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:35.728073 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:35.728187 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:35.728680 (+ 493us) server_negotiation.cc:299] Negotiation successful
0620 20:08:35.728846 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260620 20:08:35.731011 6161 tablet_service.cc:2908] Rejecting scan request for tablet 8118f7f64ab2437f99323e5f446d71c2: Uninitialized: clean time has not yet been initialized
I20260620 20:08:35.733529 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 6118
W20260620 20:08:35.737313 6382 tablet.cc:2369] T 8118f7f64ab2437f99323e5f446d71c2 P 1312405dc8264303840abfe892b162a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:08:35.738788 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 6252
I20260620 20:08:35.743610 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 6386
I20260620 20:08:35.748483 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 6024
2026-06-20T20:08:35Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2396 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:08:35 dist-test-slave-l8h4 krb5kdc[6556](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:08:35 dist-test-slave-l8h4 krb5kdc[6556](info): set up 2 sockets
Jun 20 20:08:35 dist-test-slave-l8h4 krb5kdc[6556](info): commencing operation
krb5kdc: starting...
W20260620 20:08:36.726967 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:08:36 dist-test-slave-l8h4 krb5kdc[6556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:08:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:08:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:08:36.858801 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:39281
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:44997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:39281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:36.980880 6569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:36.981189 6569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:36.981257 6569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:36.985603 6569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:08:36.985697 6569 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 20:08:36.985724 6569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:36.985749 6569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:36.985770 6569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:08:36.985944 6569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:08:36.991914 6569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:44997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:39281
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:39281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:36.993067 6569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:36.994118 6569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 20:08:37.006035 6569 server_base.cc:1034] running on GCE node
W20260620 20:08:37.006047 6574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:37.006002 6575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:37.006772 6577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:37.007606 6569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:37.009760 6569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:37.013134 6569 hybrid_clock.cc:648] HybridClock initialized: now 1781986117013092 us; error 67 us; skew 500 ppm
Jun 20 20:08:37 dist-test-slave-l8h4 krb5kdc[6556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986117, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:37.017666 6569 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:08:37.019634 6569 webserver.cc:460] Webserver started at http://127.5.91.190:42113/ using document root <none> and password file <none>
I20260620 20:08:37.020305 6569 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:37.020382 6569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:37.020598 6569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:37.022779 6569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "87ced1148c5149c996424e16bbafd4f6"
format_stamp: "Formatted at 2026-06-20 20:08:37 on dist-test-slave-l8h4"
server_key: "3a00f6cc9b5de921fd2ce921efdda0a4"
server_key_iv: "b813ded559fc7154a4d3afce1bbec669"
server_key_version: "encryptionkey@0"
I20260620 20:08:37.023343 6569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "87ced1148c5149c996424e16bbafd4f6"
format_stamp: "Formatted at 2026-06-20 20:08:37 on dist-test-slave-l8h4"
server_key: "3a00f6cc9b5de921fd2ce921efdda0a4"
server_key_iv: "b813ded559fc7154a4d3afce1bbec669"
server_key_version: "encryptionkey@0"
I20260620 20:08:37.027235 6569 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260620 20:08:37.030583 6584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:37.032661 6569 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260620 20:08:37.032859 6569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "87ced1148c5149c996424e16bbafd4f6"
format_stamp: "Formatted at 2026-06-20 20:08:37 on dist-test-slave-l8h4"
server_key: "3a00f6cc9b5de921fd2ce921efdda0a4"
server_key_iv: "b813ded559fc7154a4d3afce1bbec669"
server_key_version: "encryptionkey@0"
I20260620 20:08:37.033022 6569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:37.061507 6569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:37.064862 6569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:37.065068 6569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:37.074241 6569 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:39281
I20260620 20:08:37.074497 6646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:39281 every 8 connection(s)
I20260620 20:08:37.075261 6569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:08:37.077338 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 6569
I20260620 20:08:37.077479 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:08:37.077713 5486 external_mini_cluster.cc:1428] Setting key 102adce6b177c30bd706c30bc5f78a8e
I20260620 20:08:37.078299 6647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:37.085364 6647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6: Bootstrap starting.
I20260620 20:08:37.087991 6647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6: Neither blocks nor log segments found. Creating new log.
Jun 20 20:08:37 dist-test-slave-l8h4 krb5kdc[6556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:37.088804 6647 log.cc:824] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:37.091394 6647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6: No bootstrap required, opened a new log
I20260620 20:08:37.095937 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:37.079186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51082 (local address 127.5.91.190:39281)
0620 20:08:37.079631 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:37.079650 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:37.079692 (+ 42us) server_negotiation.cc:407] Connection header received
0620 20:08:37.080534 (+ 842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:37.080559 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:37.081001 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:37.081389 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:37.082327 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.083825 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.085171 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.085864 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.086022 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:37.088896 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:37.088903 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:37.088921 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:37.088958 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:37.091680 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:37.092167 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:37.092173 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:37.092179 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:37.092318 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:37.092615 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:37.092618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:37.092620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:37.093140 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:37.093337 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:37.094025 (+ 688us) server_negotiation.cc:299] Negotiation successful
0620 20:08:37.094402 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":142,"threads_started":1}
I20260620 20:08:37.096124 6647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ced1148c5149c996424e16bbafd4f6" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39281 } }
I20260620 20:08:37.097280 6647 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:37.097776 6647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87ced1148c5149c996424e16bbafd4f6, State: Initialized, Role: FOLLOWER
I20260620 20:08:37.098757 6647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ced1148c5149c996424e16bbafd4f6" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39281 } }
I20260620 20:08:37.098959 6647 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:08:37.099018 6647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:08:37.099108 6647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:37.100203 6647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ced1148c5149c996424e16bbafd4f6" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39281 } }
I20260620 20:08:37.100550 6647 leader_election.cc:302] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87ced1148c5149c996424e16bbafd4f6; no voters:
I20260620 20:08:37.100952 6647 leader_election.cc:288] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:08:37.101063 6652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:37.101279 6652 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6 [term 1 LEADER]: Becoming Leader. State: Replica: 87ced1148c5149c996424e16bbafd4f6, State: Running, Role: LEADER
I20260620 20:08:37.101959 6652 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ced1148c5149c996424e16bbafd4f6" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39281 } }
I20260620 20:08:37.102293 6647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:08:37.104215 6653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87ced1148c5149c996424e16bbafd4f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ced1148c5149c996424e16bbafd4f6" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39281 } } }
I20260620 20:08:37.104158 6654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87ced1148c5149c996424e16bbafd4f6. Latest consensus state: current_term: 1 leader_uuid: "87ced1148c5149c996424e16bbafd4f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ced1148c5149c996424e16bbafd4f6" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39281 } } }
I20260620 20:08:37.104444 6653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6 [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:37.104455 6654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6 [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:37.104992 6667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:08:37.108213 6667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:08:37.115259 6667 catalog_manager.cc:1347] Generated new cluster ID: 9acbb8ae105c48b6a21b47cf0c355dc4
I20260620 20:08:37.115350 6667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:08:37.127638 6667 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:08:37.128953 6667 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:08:37.141585 6667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6: Generated new TSK 0
I20260620 20:08:37.142432 6667 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.91.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
I20260620 20:08:37.181627 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:39281
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:44997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:37.311493 6673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:37.311779 6673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:37.311826 6673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:37.315974 6673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:37.316071 6673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:37.316114 6673 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 20:08:37.316147 6673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:08:37.321705 6673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:44997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.91.190:39281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:37.322798 6673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:37.323830 6673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:37.338260 6679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:37.338260 6681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:37.338390 6678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:37.338761 6673 server_base.cc:1034] running on GCE node
I20260620 20:08:37.339252 6673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:37.339954 6673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:37.341306 6673 hybrid_clock.cc:648] HybridClock initialized: now 1781986117341250 us; error 89 us; skew 500 ppm
Jun 20 20:08:37 dist-test-slave-l8h4 krb5kdc[6556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986117, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:37.344543 6673 init.cc:377] Logged in from keytab as kudu/127.5.91.129@KRBTEST.COM (short username kudu)
I20260620 20:08:37.345572 6673 webserver.cc:460] Webserver started at http://127.5.91.129:37943/ using document root <none> and password file <none>
I20260620 20:08:37.346163 6673 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:37.346247 6673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:37.346509 6673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:37.348835 6673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "68c15af52104439ca89aa126ebaa8661"
format_stamp: "Formatted at 2026-06-20 20:08:37 on dist-test-slave-l8h4"
server_key: "030d3f3af41a3ab2c6ddd29fd9402e0b"
server_key_iv: "c7b2ac8e48a6ee3d4a87f39340587875"
server_key_version: "encryptionkey@0"
I20260620 20:08:37.349390 6673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "68c15af52104439ca89aa126ebaa8661"
format_stamp: "Formatted at 2026-06-20 20:08:37 on dist-test-slave-l8h4"
server_key: "030d3f3af41a3ab2c6ddd29fd9402e0b"
server_key_iv: "c7b2ac8e48a6ee3d4a87f39340587875"
server_key_version: "encryptionkey@0"
I20260620 20:08:37.354601 6673 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260620 20:08:37.357856 6688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:37.359714 6673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:08:37.359864 6673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "68c15af52104439ca89aa126ebaa8661"
format_stamp: "Formatted at 2026-06-20 20:08:37 on dist-test-slave-l8h4"
server_key: "030d3f3af41a3ab2c6ddd29fd9402e0b"
server_key_iv: "c7b2ac8e48a6ee3d4a87f39340587875"
server_key_version: "encryptionkey@0"
I20260620 20:08:37.359984 6673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:37.384608 6673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:37.387835 6673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:37.388019 6673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:37.389524 6673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:37.389573 6673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:37.389626 6673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:37.389662 6673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:37.399477 6673 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:36633
I20260620 20:08:37.399566 6803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:36633 every 8 connection(s)
I20260620 20:08:37.400519 6673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
Jun 20 20:08:37 dist-test-slave-l8h4 krb5kdc[6556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986117, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:37.405907 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:37.390483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:58291 (local address 127.5.91.190:39281)
0620 20:08:37.390610 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:37.390613 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:37.392033 (+ 1420us) server_negotiation.cc:407] Connection header received
0620 20:08:37.394327 (+ 2294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:37.394331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:37.394398 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:37.394502 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:37.396225 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.397033 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.398646 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.398963 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.399080 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:37.402386 (+ 3306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:37.402389 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:37.402413 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:37.402445 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:37.403870 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:37.404405 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:37.404408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:37.404409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:37.404480 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:37.404841 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:37.404844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:37.404845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:37.405045 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:37.405127 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:37.405638 (+ 511us) server_negotiation.cc:299] Negotiation successful
0620 20:08:37.405764 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 20:08:37.406724 6697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:37.391046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:39281 (local address 127.5.91.129:58291)
0620 20:08:37.391869 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:37.391932 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:37.394037 (+ 2105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:37.394788 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:37.394800 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:37.395347 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:37.396037 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:37.396057 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.397167 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.397170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:37.398528 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:37.398538 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.399270 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.399276 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:37.399665 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:37.400551 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:37.400573 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:37.402200 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:37.404042 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:37.404048 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:37.404059 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:37.404310 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:37.404577 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:37.404580 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:37.404582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:37.404722 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:37.405128 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:37.405133 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:37.405387 (+ 254us) client_negotiation.cc:769] Sending connection context
0620 20:08:37.405540 (+ 153us) client_negotiation.cc:240] Negotiation successful
0620 20:08:37.405798 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":149,"threads_started":1}
I20260620 20:08:37.408115 6804 heartbeater.cc:347] Connected to a master server at 127.5.91.190:39281
I20260620 20:08:37.408272 6804 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:37.408823 6804 heartbeater.cc:511] Master 127.5.91.190:39281 requested a full tablet report, sending...
I20260620 20:08:37.409938 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 6673
I20260620 20:08:37.410207 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:08:37.410518 5486 external_mini_cluster.cc:1428] Setting key 29271510de301098ecf7f8b5f36a0421
I20260620 20:08:37.410830 6600 ts_manager.cc:194] Registered new tserver with Master: 68c15af52104439ca89aa126ebaa8661 (127.5.91.129:36633)
I20260620 20:08:37.412171 6600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:58291
I20260620 20:08:37.418695 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:37.410090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:49077 (local address 127.5.91.190:39281)
0620 20:08:37.410220 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:37.410225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:37.410422 (+ 197us) server_negotiation.cc:407] Connection header received
0620 20:08:37.410517 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:37.410520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:37.410572 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:37.410632 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:37.411078 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.411848 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.412874 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.413178 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.413293 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:37.414628 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:37.414632 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:37.414634 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:37.414658 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:37.416596 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:37.417221 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:37.417225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:37.417226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:37.417311 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:37.417571 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:37.417574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:37.417576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:37.417858 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:37.417997 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:37.418374 (+ 377us) server_negotiation.cc:299] Negotiation successful
0620 20:08:37.418526 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 20:08:37.418771 6810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:37.410013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:39281 (local address 127.5.91.129:49077)
0620 20:08:37.410346 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:37.410358 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:37.410445 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:37.410656 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:37.410659 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:37.410850 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:37.410967 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:37.410974 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.411977 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.411981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:37.412759 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:37.412770 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.413409 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.413412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:37.413504 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:37.414021 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:37.414036 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:37.414490 (+ 454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:37.416845 (+ 2355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:37.416849 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:37.416851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:37.417104 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:37.417399 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:37.417402 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:37.417404 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:37.417487 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:37.418042 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:37.418046 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:37.418169 (+ 123us) client_negotiation.cc:769] Sending connection context
0620 20:08:37.418351 (+ 182us) client_negotiation.cc:240] Negotiation successful
0620 20:08:37.418586 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for kudu/127.5.91.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
I20260620 20:08:37.436882 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:39281
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:44997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:37.550872 6813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:37.551151 6813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:37.551199 6813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:37.555791 6813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:37.555861 6813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:37.555902 6813 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 20:08:37.555933 6813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:08:37.561703 6813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:44997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.91.190:39281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:37.562791 6813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:37.563822 6813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:37.573694 6818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:37.573743 6819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:37.573765 6813 server_base.cc:1034] running on GCE node
W20260620 20:08:37.573704 6821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:37.574331 6813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:37.575016 6813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:37.576184 6813 hybrid_clock.cc:648] HybridClock initialized: now 1781986117576154 us; error 39 us; skew 500 ppm
Jun 20 20:08:37 dist-test-slave-l8h4 krb5kdc[6556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986117, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:37.578999 6813 init.cc:377] Logged in from keytab as kudu/127.5.91.130@KRBTEST.COM (short username kudu)
I20260620 20:08:37.580113 6813 webserver.cc:460] Webserver started at http://127.5.91.130:38327/ using document root <none> and password file <none>
I20260620 20:08:37.580807 6813 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:37.580895 6813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:37.581072 6813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:37.583132 6813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "0e5e02c62e0f485491831f0ab22e0741"
format_stamp: "Formatted at 2026-06-20 20:08:37 on dist-test-slave-l8h4"
server_key: "5360e2dc24a516135fdbc54588064422"
server_key_iv: "6379e75550d54994394ea3e5aa02471d"
server_key_version: "encryptionkey@0"
I20260620 20:08:37.583635 6813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "0e5e02c62e0f485491831f0ab22e0741"
format_stamp: "Formatted at 2026-06-20 20:08:37 on dist-test-slave-l8h4"
server_key: "5360e2dc24a516135fdbc54588064422"
server_key_iv: "6379e75550d54994394ea3e5aa02471d"
server_key_version: "encryptionkey@0"
I20260620 20:08:37.587329 6813 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260620 20:08:37.589898 6828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:37.591838 6813 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260620 20:08:37.591949 6813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "0e5e02c62e0f485491831f0ab22e0741"
format_stamp: "Formatted at 2026-06-20 20:08:37 on dist-test-slave-l8h4"
server_key: "5360e2dc24a516135fdbc54588064422"
server_key_iv: "6379e75550d54994394ea3e5aa02471d"
server_key_version: "encryptionkey@0"
I20260620 20:08:37.592061 6813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:37.610956 6813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:37.614486 6813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:37.614655 6813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:37.616061 6813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:37.616110 6813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:37.616149 6813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:37.616166 6813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 20:08:37 dist-test-slave-l8h4 krb5kdc[6556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986117, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:37.628286 6813 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:40477
I20260620 20:08:37.628345 6943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:40477 every 8 connection(s)
I20260620 20:08:37.629146 6813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:08:37.630494 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:37.616911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:33457 (local address 127.5.91.190:39281)
0620 20:08:37.617059 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:37.617064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:37.617929 (+ 865us) server_negotiation.cc:407] Connection header received
0620 20:08:37.618984 (+ 1055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:37.618988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:37.619045 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:37.619117 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:37.620263 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.620923 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.622056 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.622348 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.622443 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:37.626399 (+ 3956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:37.626402 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:37.626412 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:37.626438 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:37.628035 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:37.628635 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:37.628638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:37.628641 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:37.628719 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:37.629033 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:37.629037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:37.629039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:37.629226 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:37.629314 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:37.630262 (+ 948us) server_negotiation.cc:299] Negotiation successful
0620 20:08:37.630364 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 20:08:37.630954 6837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:37.617319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:39281 (local address 127.5.91.130:33457)
0620 20:08:37.617767 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:37.617815 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:37.618761 (+ 946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:37.619287 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:37.619294 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:37.619613 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:37.620131 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:37.620145 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.621036 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.621039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:37.621942 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:37.621951 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.622496 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.622500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:37.622687 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:37.623318 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:37.623343 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:37.626190 (+ 2847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:37.628178 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:37.628184 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:37.628200 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:37.628540 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:37.628801 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:37.628804 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:37.628807 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:37.628946 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:37.629321 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:37.629328 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:37.629638 (+ 310us) client_negotiation.cc:769] Sending connection context
0620 20:08:37.629853 (+ 215us) client_negotiation.cc:240] Negotiation successful
0620 20:08:37.630126 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":111,"threads_started":1}
I20260620 20:08:37.632153 6944 heartbeater.cc:347] Connected to a master server at 127.5.91.190:39281
I20260620 20:08:37.632280 6944 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:37.632730 6944 heartbeater.cc:511] Master 127.5.91.190:39281 requested a full tablet report, sending...
I20260620 20:08:37.633652 6601 ts_manager.cc:194] Registered new tserver with Master: 0e5e02c62e0f485491831f0ab22e0741 (127.5.91.130:40477)
I20260620 20:08:37.634367 6601 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:33457
I20260620 20:08:37.634460 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 6813
I20260620 20:08:37.634565 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:08:37.634816 5486 external_mini_cluster.cc:1428] Setting key 794ac8f60e8f3c3975f1ef6fa22c6e08
I20260620 20:08:37.644403 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:37.634098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:56819 (local address 127.5.91.190:39281)
0620 20:08:37.634216 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:37.634219 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:37.636741 (+ 2522us) server_negotiation.cc:407] Connection header received
0620 20:08:37.636793 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:37.636796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:37.636845 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:37.636953 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:37.637319 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.637974 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.639074 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.639293 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.639384 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:37.641174 (+ 1790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:37.641177 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:37.641179 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:37.641200 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:37.642616 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:37.643150 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:37.643153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:37.643155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:37.643244 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:37.643550 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:37.643553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:37.643554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:37.643760 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:37.643832 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:37.644159 (+ 327us) server_negotiation.cc:299] Negotiation successful
0620 20:08:37.644280 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 20:08:37.644451 6950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:37.634045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:39281 (local address 127.5.91.130:56819)
0620 20:08:37.636594 (+ 2549us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:37.636607 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:37.636687 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:37.636965 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:37.636968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:37.637132 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:37.637224 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:37.637229 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.638099 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.638102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:37.638956 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:37.638966 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.639399 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.639403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:37.639528 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:37.640057 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:37.640071 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:37.640746 (+ 675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:37.642743 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:37.642747 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:37.642749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:37.643016 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:37.643340 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:37.643343 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:37.643345 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:37.643451 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:37.643852 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:37.643855 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:37.643956 (+ 101us) client_negotiation.cc:769] Sending connection context
0620 20:08:37.644085 (+ 129us) client_negotiation.cc:240] Negotiation successful
0620 20:08:37.644247 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2484,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for kudu/127.5.91.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
I20260620 20:08:37.666262 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:39281
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:44997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:37.785394 6953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:37.785672 6953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:37.785734 6953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:37.789813 6953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:37.789882 6953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:37.789938 6953 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 20:08:37.789994 6953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:08:37.795513 6953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:44997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.91.190:39281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:37.796633 6953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:37.797628 6953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:37.808010 6958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:37.808012 6959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:37.808248 6961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:37.808298 6953 server_base.cc:1034] running on GCE node
I20260620 20:08:37.808879 6953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:37.809522 6953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:37.810719 6953 hybrid_clock.cc:648] HybridClock initialized: now 1781986117810680 us; error 52 us; skew 500 ppm
Jun 20 20:08:37 dist-test-slave-l8h4 krb5kdc[6556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986117, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:37.813557 6953 init.cc:377] Logged in from keytab as kudu/127.5.91.131@KRBTEST.COM (short username kudu)
I20260620 20:08:37.814430 6953 webserver.cc:460] Webserver started at http://127.5.91.131:42243/ using document root <none> and password file <none>
I20260620 20:08:37.814972 6953 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:37.815037 6953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:37.815218 6953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:37.817313 6953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "f80bb261c5594985aee14e2777537a5b"
format_stamp: "Formatted at 2026-06-20 20:08:37 on dist-test-slave-l8h4"
server_key: "3bec757ff0ba48d310099b9b4e29c6b0"
server_key_iv: "181fb3b642cc56701cf3436078d6bf6b"
server_key_version: "encryptionkey@0"
I20260620 20:08:37.818019 6953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "f80bb261c5594985aee14e2777537a5b"
format_stamp: "Formatted at 2026-06-20 20:08:37 on dist-test-slave-l8h4"
server_key: "3bec757ff0ba48d310099b9b4e29c6b0"
server_key_iv: "181fb3b642cc56701cf3436078d6bf6b"
server_key_version: "encryptionkey@0"
I20260620 20:08:37.822467 6953 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260620 20:08:37.825286 6968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:37.826972 6953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 20:08:37.827116 6953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "f80bb261c5594985aee14e2777537a5b"
format_stamp: "Formatted at 2026-06-20 20:08:37 on dist-test-slave-l8h4"
server_key: "3bec757ff0ba48d310099b9b4e29c6b0"
server_key_iv: "181fb3b642cc56701cf3436078d6bf6b"
server_key_version: "encryptionkey@0"
I20260620 20:08:37.827225 6953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:37.839748 6953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:37.842839 6953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:37.843006 6953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:37.844590 6953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:37.844657 6953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:37.844707 6953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:37.844748 6953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 20:08:37 dist-test-slave-l8h4 krb5kdc[6556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986117, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:37.855048 6953 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:36731
I20260620 20:08:37.855101 7083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:36731 every 8 connection(s)
I20260620 20:08:37.855988 6953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:08:37.859200 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:37.845484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:36177 (local address 127.5.91.190:39281)
0620 20:08:37.845636 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:37.845642 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:37.846873 (+ 1231us) server_negotiation.cc:407] Connection header received
0620 20:08:37.848022 (+ 1149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:37.848025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:37.848104 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:37.848204 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:37.849646 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.850484 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.851771 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.852115 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.852225 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:37.855528 (+ 3303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:37.855532 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:37.855535 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:37.855563 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:37.857018 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:37.857492 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:37.857495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:37.857496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:37.857570 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:37.857837 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:37.857838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:37.857839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:37.858007 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:37.858078 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:37.858833 (+ 755us) server_negotiation.cc:299] Negotiation successful
0620 20:08:37.859032 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 20:08:37.859681 6977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:37.845916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:39281 (local address 127.5.91.131:36177)
0620 20:08:37.846534 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:37.846643 (+ 109us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:37.847791 (+ 1148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:37.848429 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:37.848436 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:37.848824 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:37.849484 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:37.849504 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.850611 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.850614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:37.851668 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:37.851677 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.852275 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.852278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:37.852430 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:37.853613 (+ 1183us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:37.853637 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:37.855352 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:37.857120 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:37.857124 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:37.857134 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:37.857402 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:37.857631 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:37.857633 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:37.857634 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:37.857748 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:37.858182 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:37.858189 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:37.858463 (+ 274us) client_negotiation.cc:769] Sending connection context
0620 20:08:37.858631 (+ 168us) client_negotiation.cc:240] Negotiation successful
0620 20:08:37.858877 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":113,"threads_started":1}
I20260620 20:08:37.860949 7084 heartbeater.cc:347] Connected to a master server at 127.5.91.190:39281
I20260620 20:08:37.861095 7084 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:37.861582 7084 heartbeater.cc:511] Master 127.5.91.190:39281 requested a full tablet report, sending...
I20260620 20:08:37.862571 6600 ts_manager.cc:194] Registered new tserver with Master: f80bb261c5594985aee14e2777537a5b (127.5.91.131:36731)
I20260620 20:08:37.863263 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 6953
I20260620 20:08:37.863293 6600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:36177
I20260620 20:08:37.863379 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:08:37.863660 5486 external_mini_cluster.cc:1428] Setting key 11c65f55da9062f93a23b1b16403ec9a
I20260620 20:08:37.865578 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 20:08:37.870541 7090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:37.863246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:39281 (local address 127.5.91.131:56717)
0620 20:08:37.863566 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:37.863580 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:37.863673 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:37.863904 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:37.863907 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:37.864099 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:37.864213 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:37.864221 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.865234 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.865237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:37.865913 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:37.865922 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.866393 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.866397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:37.866483 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:37.866938 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:37.866951 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:37.867249 (+ 298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:37.868965 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:37.868968 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:37.868970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:37.869249 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:37.869525 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:37.869528 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:37.869530 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:37.869626 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:37.869984 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:37.869987 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:37.870078 (+ 91us) client_negotiation.cc:769] Sending connection context
0620 20:08:37.870194 (+ 116us) client_negotiation.cc:240] Negotiation successful
0620 20:08:37.870363 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":100,"threads_started":1}
I20260620 20:08:37.870532 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:37.863305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:56717 (local address 127.5.91.190:39281)
0620 20:08:37.863411 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:37.863414 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:37.863636 (+ 222us) server_negotiation.cc:407] Connection header received
0620 20:08:37.863761 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:37.863764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:37.863809 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:37.863881 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:37.864371 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.865108 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.866016 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.866291 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.866371 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:37.867379 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:37.867383 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:37.867385 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:37.867409 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:37.868858 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:37.869361 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:37.869364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:37.869366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:37.869444 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:37.869699 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:37.869702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:37.869705 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:37.869899 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:37.869985 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:37.870243 (+ 258us) server_negotiation.cc:299] Negotiation successful
0620 20:08:37.870359 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Jun 20 20:08:37 dist-test-slave-l8h4 krb5kdc[6556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986117, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 20:08:37 dist-test-slave-l8h4 krb5kdc[6556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986117, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:37.893885 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:37.884496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51096 (local address 127.5.91.190:39281)
0620 20:08:37.884805 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:37.884808 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:37.884820 (+ 12us) server_negotiation.cc:407] Connection header received
0620 20:08:37.884863 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:37.884866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:37.884906 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:37.884976 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:37.885636 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.886169 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.887375 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.887594 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.887683 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:37.890265 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:37.890268 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:37.890270 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:37.890296 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:37.892056 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:37.892559 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:37.892563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:37.892565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:37.892660 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:37.893010 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:37.893013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:37.893015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:37.893223 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:37.893339 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:37.893647 (+ 308us) server_negotiation.cc:299] Negotiation successful
0620 20:08:37.893758 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217}
I20260620 20:08:37.898559 6600 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:08:37.902910 6600 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:08:37.917972 7101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:37.912429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:36633 (local address 127.0.0.1:38416)
0620 20:08:37.912817 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:37.912869 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:37.912988 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:37.913684 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:37.913690 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:37.913711 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:37.913849 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:37.913856 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.915197 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.915200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:37.916367 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:37.916376 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.917441 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.917444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:37.917552 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:37.917587 (+ 35us) client_negotiation.cc:769] Sending connection context
0620 20:08:37.917691 (+ 104us) client_negotiation.cc:240] Negotiation successful
0620 20:08:37.917819 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":94,"threads_started":1}
I20260620 20:08:37.917979 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:37.912585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38416 (local address 127.5.91.129:36633)
0620 20:08:37.912930 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:37.912938 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:37.912963 (+ 25us) server_negotiation.cc:407] Connection header received
0620 20:08:37.913154 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:37.913160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:37.913377 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:37.913523 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:37.913984 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.915078 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.916491 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.917341 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.917440 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:37.917593 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:37.917727 (+ 134us) server_negotiation.cc:299] Negotiation successful
0620 20:08:37.917827 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":72,"threads_started":1}
I20260620 20:08:37.918378 7104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:37.912703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:40477 (local address 127.0.0.1:37450)
0620 20:08:37.913545 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:37.913562 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:37.913660 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:37.914457 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:37.914460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:37.914477 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:37.914566 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:37.914572 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.915727 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.915730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:37.916909 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:37.916919 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.917981 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.917985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:37.918071 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:37.918096 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 20:08:37.918170 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 20:08:37.918248 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":26,"thread_start_us":124,"threads_started":1}
I20260620 20:08:37.918848 7103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:37.913153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:36731 (local address 127.0.0.1:40186)
0620 20:08:37.913696 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:37.913719 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:37.913789 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:37.914686 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:37.914689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:37.914702 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:37.914778 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:37.914782 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.916012 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.916015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:37.917145 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:37.917153 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.918412 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.918416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:37.918575 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:37.918599 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 20:08:37.918652 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 20:08:37.918730 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":24,"thread_start_us":80,"threads_started":1}
I20260620 20:08:37.919066 7105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:37.913348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37450 (local address 127.5.91.130:40477)
0620 20:08:37.914003 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:37.914014 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:37.914036 (+ 22us) server_negotiation.cc:407] Connection header received
0620 20:08:37.914101 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:37.914106 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:37.914310 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:37.914694 (+ 384us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:37.914838 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.915620 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.917035 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.917849 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.918632 (+ 783us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:37.918795 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:37.918842 (+ 47us) server_negotiation.cc:299] Negotiation successful
0620 20:08:37.918937 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":113,"threads_started":1}
I20260620 20:08:37.920084 7106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:37.913380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40186 (local address 127.5.91.131:36731)
0620 20:08:37.913876 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:37.913887 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:37.913909 (+ 22us) server_negotiation.cc:407] Connection header received
0620 20:08:37.913981 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:37.913986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:37.914434 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:37.914576 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:37.914889 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.915898 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.917264 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:37.918009 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:37.919082 (+ 1073us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:37.919806 (+ 724us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:37.919858 (+ 52us) server_negotiation.cc:299] Negotiation successful
0620 20:08:37.919950 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":136,"threads_started":1}
I20260620 20:08:37.921212 6737 tablet_service.cc:1467] Processing CreateTablet for tablet 11b5b49c187b482fb71fc3bdceb6a95f (DEFAULT_TABLE table=test-table [id=079fcc4b3c674e2b858f4759e14237a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:37.921379 6878 tablet_service.cc:1467] Processing CreateTablet for tablet 11b5b49c187b482fb71fc3bdceb6a95f (DEFAULT_TABLE table=test-table [id=079fcc4b3c674e2b858f4759e14237a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:37.922252 6878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11b5b49c187b482fb71fc3bdceb6a95f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:37.922256 6737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11b5b49c187b482fb71fc3bdceb6a95f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:37.924929 7018 tablet_service.cc:1467] Processing CreateTablet for tablet 11b5b49c187b482fb71fc3bdceb6a95f (DEFAULT_TABLE table=test-table [id=079fcc4b3c674e2b858f4759e14237a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:37.926105 7018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11b5b49c187b482fb71fc3bdceb6a95f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:37.928376 7107 tablet_bootstrap.cc:492] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661: Bootstrap starting.
I20260620 20:08:37.930292 7108 tablet_bootstrap.cc:492] T 11b5b49c187b482fb71fc3bdceb6a95f P 0e5e02c62e0f485491831f0ab22e0741: Bootstrap starting.
I20260620 20:08:37.931493 7107 tablet_bootstrap.cc:654] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:37.932324 7108 tablet_bootstrap.cc:654] T 11b5b49c187b482fb71fc3bdceb6a95f P 0e5e02c62e0f485491831f0ab22e0741: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:37.932341 7107 log.cc:824] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:37.933089 7108 log.cc:824] T 11b5b49c187b482fb71fc3bdceb6a95f P 0e5e02c62e0f485491831f0ab22e0741: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:37.934222 7107 tablet_bootstrap.cc:492] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661: No bootstrap required, opened a new log
I20260620 20:08:37.934276 7111 tablet_bootstrap.cc:492] T 11b5b49c187b482fb71fc3bdceb6a95f P f80bb261c5594985aee14e2777537a5b: Bootstrap starting.
I20260620 20:08:37.934492 7107 ts_tablet_manager.cc:1397] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260620 20:08:37.935257 7108 tablet_bootstrap.cc:492] T 11b5b49c187b482fb71fc3bdceb6a95f P 0e5e02c62e0f485491831f0ab22e0741: No bootstrap required, opened a new log
I20260620 20:08:37.935489 7108 ts_tablet_manager.cc:1397] T 11b5b49c187b482fb71fc3bdceb6a95f P 0e5e02c62e0f485491831f0ab22e0741: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260620 20:08:37.936403 7111 tablet_bootstrap.cc:654] T 11b5b49c187b482fb71fc3bdceb6a95f P f80bb261c5594985aee14e2777537a5b: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:37.937269 7111 log.cc:824] T 11b5b49c187b482fb71fc3bdceb6a95f P f80bb261c5594985aee14e2777537a5b: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:37.938576 7107 raft_consensus.cc:348] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } }
I20260620 20:08:37.938854 7107 raft_consensus.cc:374] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:37.938907 7107 raft_consensus.cc:729] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c15af52104439ca89aa126ebaa8661, State: Initialized, Role: FOLLOWER
I20260620 20:08:37.939409 7107 consensus_queue.cc:260] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } }
I20260620 20:08:37.939831 7111 tablet_bootstrap.cc:492] T 11b5b49c187b482fb71fc3bdceb6a95f P f80bb261c5594985aee14e2777537a5b: No bootstrap required, opened a new log
I20260620 20:08:37.939713 7108 raft_consensus.cc:348] T 11b5b49c187b482fb71fc3bdceb6a95f P 0e5e02c62e0f485491831f0ab22e0741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } }
I20260620 20:08:37.939985 7108 raft_consensus.cc:374] T 11b5b49c187b482fb71fc3bdceb6a95f P 0e5e02c62e0f485491831f0ab22e0741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:37.940021 7111 ts_tablet_manager.cc:1397] T 11b5b49c187b482fb71fc3bdceb6a95f P f80bb261c5594985aee14e2777537a5b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 20:08:37.940040 7108 raft_consensus.cc:729] T 11b5b49c187b482fb71fc3bdceb6a95f P 0e5e02c62e0f485491831f0ab22e0741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e5e02c62e0f485491831f0ab22e0741, State: Initialized, Role: FOLLOWER
I20260620 20:08:37.940276 6804 heartbeater.cc:503] Master 127.5.91.190:39281 was elected leader, sending a full tablet report...
I20260620 20:08:37.940407 7107 ts_tablet_manager.cc:1428] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260620 20:08:37.941437 7108 consensus_queue.cc:260] T 11b5b49c187b482fb71fc3bdceb6a95f P 0e5e02c62e0f485491831f0ab22e0741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } }
I20260620 20:08:37.942456 7108 ts_tablet_manager.cc:1428] T 11b5b49c187b482fb71fc3bdceb6a95f P 0e5e02c62e0f485491831f0ab22e0741: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260620 20:08:37.942806 6944 heartbeater.cc:503] Master 127.5.91.190:39281 was elected leader, sending a full tablet report...
I20260620 20:08:37.943641 7111 raft_consensus.cc:348] T 11b5b49c187b482fb71fc3bdceb6a95f P f80bb261c5594985aee14e2777537a5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } }
I20260620 20:08:37.943897 7111 raft_consensus.cc:374] T 11b5b49c187b482fb71fc3bdceb6a95f P f80bb261c5594985aee14e2777537a5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:37.943964 7111 raft_consensus.cc:729] T 11b5b49c187b482fb71fc3bdceb6a95f P f80bb261c5594985aee14e2777537a5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f80bb261c5594985aee14e2777537a5b, State: Initialized, Role: FOLLOWER
I20260620 20:08:37.944360 7111 consensus_queue.cc:260] T 11b5b49c187b482fb71fc3bdceb6a95f P f80bb261c5594985aee14e2777537a5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } }
I20260620 20:08:37.945115 7084 heartbeater.cc:503] Master 127.5.91.190:39281 was elected leader, sending a full tablet report...
I20260620 20:08:37.945326 7111 ts_tablet_manager.cc:1428] T 11b5b49c187b482fb71fc3bdceb6a95f P f80bb261c5594985aee14e2777537a5b: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260620 20:08:38.015166 7113 raft_consensus.cc:482] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:08:38.015427 7113 raft_consensus.cc:504] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } }
I20260620 20:08:38.016611 7113 leader_election.cc:288] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f80bb261c5594985aee14e2777537a5b (127.5.91.131:36731), 0e5e02c62e0f485491831f0ab22e0741 (127.5.91.130:40477)
I20260620 20:08:38.022514 7105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:38.016998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:48405 (local address 127.5.91.130:40477)
0620 20:08:38.017125 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:38.017130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:38.017424 (+ 294us) server_negotiation.cc:407] Connection header received
0620 20:08:38.017535 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:38.017538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:38.017589 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:38.017691 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:38.018181 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.019218 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.021120 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.021802 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.021946 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:38.022005 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:38.022274 (+ 269us) server_negotiation.cc:299] Negotiation successful
0620 20:08:38.022379 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 20:08:38.022514 7106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:38.017026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:35683 (local address 127.5.91.131:36731)
0620 20:08:38.017150 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:38.017154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:38.017473 (+ 319us) server_negotiation.cc:407] Connection header received
0620 20:08:38.017534 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:38.017538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:38.017589 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:38.017680 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:38.018181 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.019208 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.021121 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.021802 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.021954 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:38.022005 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:38.022273 (+ 268us) server_negotiation.cc:299] Negotiation successful
0620 20:08:38.022379 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 20:08:38.022563 7117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:38.016897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:36731 (local address 127.5.91.129:35683)
0620 20:08:38.017357 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:38.017368 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:38.017458 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:38.017700 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:38.017703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:38.017742 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:38.017927 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:38.017932 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.019402 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.019406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:38.020954 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:38.020960 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.021957 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.021960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:38.022105 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:38.022170 (+ 65us) client_negotiation.cc:769] Sending connection context
0620 20:08:38.022251 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 20:08:38.022366 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":250,"spinlock_wait_cycles":30336,"thread_start_us":110,"threads_started":1}
I20260620 20:08:38.022801 7116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:38.016897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:40477 (local address 127.5.91.129:48405)
0620 20:08:38.017290 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:38.017305 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:38.017432 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:38.017700 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:38.017703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:38.017744 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:38.017916 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:38.017923 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.019402 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.019406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:38.020946 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:38.020959 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.021951 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.021955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:38.022117 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:38.022178 (+ 61us) client_negotiation.cc:769] Sending connection context
0620 20:08:38.022251 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 20:08:38.022366 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":208,"spinlock_wait_cycles":3968,"thread_start_us":115,"threads_started":1}
I20260620 20:08:38.023397 7038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11b5b49c187b482fb71fc3bdceb6a95f" candidate_uuid: "68c15af52104439ca89aa126ebaa8661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f80bb261c5594985aee14e2777537a5b" is_pre_election: true
I20260620 20:08:38.023398 6898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11b5b49c187b482fb71fc3bdceb6a95f" candidate_uuid: "68c15af52104439ca89aa126ebaa8661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e5e02c62e0f485491831f0ab22e0741" is_pre_election: true
I20260620 20:08:38.023715 7038 raft_consensus.cc:2454] T 11b5b49c187b482fb71fc3bdceb6a95f P f80bb261c5594985aee14e2777537a5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68c15af52104439ca89aa126ebaa8661 in term 0.
I20260620 20:08:38.023715 6898 raft_consensus.cc:2454] T 11b5b49c187b482fb71fc3bdceb6a95f P 0e5e02c62e0f485491831f0ab22e0741 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68c15af52104439ca89aa126ebaa8661 in term 0.
I20260620 20:08:38.024200 6690 leader_election.cc:302] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e5e02c62e0f485491831f0ab22e0741, 68c15af52104439ca89aa126ebaa8661; no voters:
I20260620 20:08:38.024466 7113 raft_consensus.cc:2781] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:08:38.024545 7113 raft_consensus.cc:482] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:08:38.024578 7113 raft_consensus.cc:3037] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:38.025638 7113 raft_consensus.cc:504] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } }
I20260620 20:08:38.025993 7113 leader_election.cc:288] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661 [CANDIDATE]: Term 1 election: Requested vote from peers f80bb261c5594985aee14e2777537a5b (127.5.91.131:36731), 0e5e02c62e0f485491831f0ab22e0741 (127.5.91.130:40477)
I20260620 20:08:38.026319 7038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11b5b49c187b482fb71fc3bdceb6a95f" candidate_uuid: "68c15af52104439ca89aa126ebaa8661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f80bb261c5594985aee14e2777537a5b"
I20260620 20:08:38.026432 7038 raft_consensus.cc:3037] T 11b5b49c187b482fb71fc3bdceb6a95f P f80bb261c5594985aee14e2777537a5b [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:38.026741 6898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11b5b49c187b482fb71fc3bdceb6a95f" candidate_uuid: "68c15af52104439ca89aa126ebaa8661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e5e02c62e0f485491831f0ab22e0741"
I20260620 20:08:38.026872 6898 raft_consensus.cc:3037] T 11b5b49c187b482fb71fc3bdceb6a95f P 0e5e02c62e0f485491831f0ab22e0741 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:38.027665 7038 raft_consensus.cc:2454] T 11b5b49c187b482fb71fc3bdceb6a95f P f80bb261c5594985aee14e2777537a5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c15af52104439ca89aa126ebaa8661 in term 1.
I20260620 20:08:38.028023 6692 leader_election.cc:302] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c15af52104439ca89aa126ebaa8661, f80bb261c5594985aee14e2777537a5b; no voters:
I20260620 20:08:38.028193 7113 raft_consensus.cc:2781] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:38.028400 7113 raft_consensus.cc:686] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661 [term 1 LEADER]: Becoming Leader. State: Replica: 68c15af52104439ca89aa126ebaa8661, State: Running, Role: LEADER
I20260620 20:08:38.028553 6898 raft_consensus.cc:2454] T 11b5b49c187b482fb71fc3bdceb6a95f P 0e5e02c62e0f485491831f0ab22e0741 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c15af52104439ca89aa126ebaa8661 in term 1.
I20260620 20:08:38.028741 7113 consensus_queue.cc:237] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } }
I20260620 20:08:38.032588 6601 catalog_manager.cc:5495] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661 reported cstate change: term changed from 0 to 1, leader changed from <none> to 68c15af52104439ca89aa126ebaa8661 (127.5.91.129). New cstate: current_term: 1 leader_uuid: "68c15af52104439ca89aa126ebaa8661" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } health_report { overall_health: UNKNOWN } } }
I20260620 20:08:38.078783 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:38.075187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38432 (local address 127.5.91.129:36633)
0620 20:08:38.075323 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:38.075328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:38.075398 (+ 70us) server_negotiation.cc:407] Connection header received
0620 20:08:38.075602 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:38.075604 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:38.075659 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:38.075737 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:08:38.076023 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.076525 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.077354 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.077568 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.077650 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:38.077959 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:08:38.078393 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:08:38.078491 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:38.078582 (+ 91us) server_negotiation.cc:299] Negotiation successful
0620 20:08:38.078641 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 20:08:38.084618 6898 raft_consensus.cc:1261] T 11b5b49c187b482fb71fc3bdceb6a95f P 0e5e02c62e0f485491831f0ab22e0741 [term 1 FOLLOWER]: Refusing update from remote peer 68c15af52104439ca89aa126ebaa8661: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:08:38.084875 7038 raft_consensus.cc:1261] T 11b5b49c187b482fb71fc3bdceb6a95f P f80bb261c5594985aee14e2777537a5b [term 1 FOLLOWER]: Refusing update from remote peer 68c15af52104439ca89aa126ebaa8661: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:08:38.085384 7113 consensus_queue.cc:1035] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:08:38.085608 7118 consensus_queue.cc:1035] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:08:38.092836 7121 mvcc.cc:205] Tried to move back new op lower bound from 7299015139667312640 to 7299015139449225216. Current Snapshot: MvccSnapshot[applied={T|T < 7299015139667312640}]
I20260620 20:08:38.104112 6601 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:08:38.104609 6601 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:08:38.107323 6601 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6: Sending DeleteTablet for 3 replicas of tablet 11b5b49c187b482fb71fc3bdceb6a95f
I20260620 20:08:38.108337 6737 tablet_service.cc:1514] Processing DeleteTablet for tablet 11b5b49c187b482fb71fc3bdceb6a95f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:38 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:38416
I20260620 20:08:38.108350 6878 tablet_service.cc:1514] Processing DeleteTablet for tablet 11b5b49c187b482fb71fc3bdceb6a95f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:38 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:37450
I20260620 20:08:38.108840 7018 tablet_service.cc:1514] Processing DeleteTablet for tablet 11b5b49c187b482fb71fc3bdceb6a95f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:38 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:40186
I20260620 20:08:38.108924 7131 tablet_replica.cc:330] T 11b5b49c187b482fb71fc3bdceb6a95f P 0e5e02c62e0f485491831f0ab22e0741: stopping tablet replica
I20260620 20:08:38.109186 7131 raft_consensus.cc:2229] T 11b5b49c187b482fb71fc3bdceb6a95f P 0e5e02c62e0f485491831f0ab22e0741 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:08:38.109282 7132 tablet_replica.cc:330] T 11b5b49c187b482fb71fc3bdceb6a95f P f80bb261c5594985aee14e2777537a5b: stopping tablet replica
I20260620 20:08:38.109496 7131 raft_consensus.cc:2258] T 11b5b49c187b482fb71fc3bdceb6a95f P 0e5e02c62e0f485491831f0ab22e0741 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:08:38.109519 7132 raft_consensus.cc:2229] T 11b5b49c187b482fb71fc3bdceb6a95f P f80bb261c5594985aee14e2777537a5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:08:38.109766 7132 raft_consensus.cc:2258] T 11b5b49c187b482fb71fc3bdceb6a95f P f80bb261c5594985aee14e2777537a5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:08:38.110385 6601 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:08:38.110816 6601 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:08:38.111065 7131 ts_tablet_manager.cc:1905] T 11b5b49c187b482fb71fc3bdceb6a95f P 0e5e02c62e0f485491831f0ab22e0741: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:08:38.111064 7132 ts_tablet_manager.cc:1905] T 11b5b49c187b482fb71fc3bdceb6a95f P f80bb261c5594985aee14e2777537a5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:08:38.113584 7132 ts_tablet_manager.cc:1918] T 11b5b49c187b482fb71fc3bdceb6a95f P f80bb261c5594985aee14e2777537a5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:08:38.113584 7131 ts_tablet_manager.cc:1918] T 11b5b49c187b482fb71fc3bdceb6a95f P 0e5e02c62e0f485491831f0ab22e0741: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:08:38.113678 7131 log.cc:1196] T 11b5b49c187b482fb71fc3bdceb6a95f P 0e5e02c62e0f485491831f0ab22e0741: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-1/wal/wals/11b5b49c187b482fb71fc3bdceb6a95f
I20260620 20:08:38.113678 7132 log.cc:1196] T 11b5b49c187b482fb71fc3bdceb6a95f P f80bb261c5594985aee14e2777537a5b: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/11b5b49c187b482fb71fc3bdceb6a95f
I20260620 20:08:38.113960 7131 ts_tablet_manager.cc:1939] T 11b5b49c187b482fb71fc3bdceb6a95f P 0e5e02c62e0f485491831f0ab22e0741: Deleting consensus metadata
I20260620 20:08:38.113960 7132 ts_tablet_manager.cc:1939] T 11b5b49c187b482fb71fc3bdceb6a95f P f80bb261c5594985aee14e2777537a5b: Deleting consensus metadata
I20260620 20:08:38.115144 6588 catalog_manager.cc:4841] TS f80bb261c5594985aee14e2777537a5b (127.5.91.131:36731): tablet 11b5b49c187b482fb71fc3bdceb6a95f (table test-table [id=079fcc4b3c674e2b858f4759e14237a6]) successfully deleted
I20260620 20:08:38.115144 6586 catalog_manager.cc:4841] TS 0e5e02c62e0f485491831f0ab22e0741 (127.5.91.130:40477): tablet 11b5b49c187b482fb71fc3bdceb6a95f (table test-table [id=079fcc4b3c674e2b858f4759e14237a6]) successfully deleted
I20260620 20:08:38.116200 6878 tablet_service.cc:1467] Processing CreateTablet for tablet 895e37059dc7403d84b66d04cf3dc8fa (DEFAULT_TABLE table=test-table [id=622f273861ce499a9ca087bd1f7f609a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:38.116436 6878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 895e37059dc7403d84b66d04cf3dc8fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:38.116513 6737 tablet_service.cc:1467] Processing CreateTablet for tablet 895e37059dc7403d84b66d04cf3dc8fa (DEFAULT_TABLE table=test-table [id=622f273861ce499a9ca087bd1f7f609a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:38.116547 7130 tablet_replica.cc:330] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661: stopping tablet replica
I20260620 20:08:38.116729 6737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 895e37059dc7403d84b66d04cf3dc8fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:38.116849 7130 raft_consensus.cc:2229] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661 [term 1 LEADER]: Raft consensus shutting down.
I20260620 20:08:38.117174 7130 raft_consensus.cc:2258] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:08:38.118647 7108 tablet_bootstrap.cc:492] T 895e37059dc7403d84b66d04cf3dc8fa P 0e5e02c62e0f485491831f0ab22e0741: Bootstrap starting.
I20260620 20:08:38.119171 7107 tablet_bootstrap.cc:492] T 895e37059dc7403d84b66d04cf3dc8fa P 68c15af52104439ca89aa126ebaa8661: Bootstrap starting.
I20260620 20:08:38.119567 7108 tablet_bootstrap.cc:654] T 895e37059dc7403d84b66d04cf3dc8fa P 0e5e02c62e0f485491831f0ab22e0741: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:38.120049 7107 tablet_bootstrap.cc:654] T 895e37059dc7403d84b66d04cf3dc8fa P 68c15af52104439ca89aa126ebaa8661: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:38.120404 7018 tablet_service.cc:1467] Processing CreateTablet for tablet 895e37059dc7403d84b66d04cf3dc8fa (DEFAULT_TABLE table=test-table [id=622f273861ce499a9ca087bd1f7f609a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:38.120657 7018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 895e37059dc7403d84b66d04cf3dc8fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:38.121941 7107 tablet_bootstrap.cc:492] T 895e37059dc7403d84b66d04cf3dc8fa P 68c15af52104439ca89aa126ebaa8661: No bootstrap required, opened a new log
I20260620 20:08:38.122030 7107 ts_tablet_manager.cc:1397] T 895e37059dc7403d84b66d04cf3dc8fa P 68c15af52104439ca89aa126ebaa8661: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260620 20:08:38.122501 7107 raft_consensus.cc:348] T 895e37059dc7403d84b66d04cf3dc8fa P 68c15af52104439ca89aa126ebaa8661 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } }
I20260620 20:08:38.122622 7107 raft_consensus.cc:374] T 895e37059dc7403d84b66d04cf3dc8fa P 68c15af52104439ca89aa126ebaa8661 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:38.122651 7107 raft_consensus.cc:729] T 895e37059dc7403d84b66d04cf3dc8fa P 68c15af52104439ca89aa126ebaa8661 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c15af52104439ca89aa126ebaa8661, State: Initialized, Role: FOLLOWER
I20260620 20:08:38.122802 7111 tablet_bootstrap.cc:492] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b: Bootstrap starting.
I20260620 20:08:38.122921 7107 consensus_queue.cc:260] T 895e37059dc7403d84b66d04cf3dc8fa P 68c15af52104439ca89aa126ebaa8661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } }
I20260620 20:08:38.123366 7107 ts_tablet_manager.cc:1428] T 895e37059dc7403d84b66d04cf3dc8fa P 68c15af52104439ca89aa126ebaa8661: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260620 20:08:38.123769 7111 tablet_bootstrap.cc:654] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:38.123821 7130 ts_tablet_manager.cc:1905] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:08:38.125183 7108 tablet_bootstrap.cc:492] T 895e37059dc7403d84b66d04cf3dc8fa P 0e5e02c62e0f485491831f0ab22e0741: No bootstrap required, opened a new log
I20260620 20:08:38.125272 7108 ts_tablet_manager.cc:1397] T 895e37059dc7403d84b66d04cf3dc8fa P 0e5e02c62e0f485491831f0ab22e0741: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260620 20:08:38.125725 7108 raft_consensus.cc:348] T 895e37059dc7403d84b66d04cf3dc8fa P 0e5e02c62e0f485491831f0ab22e0741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } }
I20260620 20:08:38.125847 7108 raft_consensus.cc:374] T 895e37059dc7403d84b66d04cf3dc8fa P 0e5e02c62e0f485491831f0ab22e0741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:38.125885 7108 raft_consensus.cc:729] T 895e37059dc7403d84b66d04cf3dc8fa P 0e5e02c62e0f485491831f0ab22e0741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e5e02c62e0f485491831f0ab22e0741, State: Initialized, Role: FOLLOWER
I20260620 20:08:38.126005 7108 consensus_queue.cc:260] T 895e37059dc7403d84b66d04cf3dc8fa P 0e5e02c62e0f485491831f0ab22e0741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } }
I20260620 20:08:38.126255 7108 ts_tablet_manager.cc:1428] T 895e37059dc7403d84b66d04cf3dc8fa P 0e5e02c62e0f485491831f0ab22e0741: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260620 20:08:38.127048 7111 tablet_bootstrap.cc:492] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b: No bootstrap required, opened a new log
I20260620 20:08:38.127139 7111 ts_tablet_manager.cc:1397] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260620 20:08:38.127571 7111 raft_consensus.cc:348] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } }
I20260620 20:08:38.127671 7111 raft_consensus.cc:374] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:38.127701 7111 raft_consensus.cc:729] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f80bb261c5594985aee14e2777537a5b, State: Initialized, Role: FOLLOWER
I20260620 20:08:38.127811 7111 consensus_queue.cc:260] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } }
I20260620 20:08:38.128069 7111 ts_tablet_manager.cc:1428] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 20:08:38.128166 7130 ts_tablet_manager.cc:1918] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:08:38.128806 7130 log.cc:1196] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-0/wal/wals/11b5b49c187b482fb71fc3bdceb6a95f
I20260620 20:08:38.129130 7130 ts_tablet_manager.cc:1939] T 11b5b49c187b482fb71fc3bdceb6a95f P 68c15af52104439ca89aa126ebaa8661: Deleting consensus metadata
W20260620 20:08:38.130362 6945 tablet.cc:2369] T 895e37059dc7403d84b66d04cf3dc8fa P 0e5e02c62e0f485491831f0ab22e0741: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:08:38.130332 6588 catalog_manager.cc:4841] TS 68c15af52104439ca89aa126ebaa8661 (127.5.91.129:36633): tablet 11b5b49c187b482fb71fc3bdceb6a95f (table test-table [id=079fcc4b3c674e2b858f4759e14237a6]) successfully deleted
W20260620 20:08:38.151899 6805 tablet.cc:2369] T 895e37059dc7403d84b66d04cf3dc8fa P 68c15af52104439ca89aa126ebaa8661: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:08:38.207906 7115 raft_consensus.cc:482] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:08:38.208065 7115 raft_consensus.cc:504] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } }
I20260620 20:08:38.209084 7115 leader_election.cc:288] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68c15af52104439ca89aa126ebaa8661 (127.5.91.129:36633), 0e5e02c62e0f485491831f0ab22e0741 (127.5.91.130:40477)
I20260620 20:08:38.213268 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:38.209421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:34457 (local address 127.5.91.129:36633)
0620 20:08:38.209552 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:38.209557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:38.209575 (+ 18us) server_negotiation.cc:407] Connection header received
0620 20:08:38.209733 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:38.209736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:38.209782 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:38.209855 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:38.210272 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.210790 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.212139 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.212715 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.212843 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:38.212959 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:38.213064 (+ 105us) server_negotiation.cc:299] Negotiation successful
0620 20:08:38.213135 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260620 20:08:38.213268 6977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:38.209323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:36633 (local address 127.5.91.131:34457)
0620 20:08:38.209499 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:38.209513 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:38.209601 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:38.209962 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:38.209965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:38.209981 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:38.210088 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:38.210095 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.210889 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.210892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:38.211994 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:38.212004 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.212851 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.212855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:38.212955 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:38.212977 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 20:08:38.213049 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 20:08:38.213124 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260620 20:08:38.213399 7105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:38.209695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:36385 (local address 127.5.91.130:40477)
0620 20:08:38.209807 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:38.209810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:38.209824 (+ 14us) server_negotiation.cc:407] Connection header received
0620 20:08:38.209985 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:38.209988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:38.210026 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:38.210085 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:38.210383 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.211157 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.212406 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.212793 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.212901 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:38.212955 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:38.213219 (+ 264us) server_negotiation.cc:299] Negotiation successful
0620 20:08:38.213297 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 20:08:38.213872 7136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:38.209349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:40477 (local address 127.5.91.131:36385)
0620 20:08:38.209735 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:38.209748 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:38.209886 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:38.210109 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:38.210111 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:38.210125 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:38.210265 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:38.210271 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.211291 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.211294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:38.212298 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:38.212305 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.213016 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.213019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:38.213101 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:38.213119 (+ 18us) client_negotiation.cc:769] Sending connection context
0620 20:08:38.213663 (+ 544us) client_negotiation.cc:240] Negotiation successful
0620 20:08:38.213735 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":82,"threads_started":1}
I20260620 20:08:38.213948 6758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "895e37059dc7403d84b66d04cf3dc8fa" candidate_uuid: "f80bb261c5594985aee14e2777537a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c15af52104439ca89aa126ebaa8661" is_pre_election: true
I20260620 20:08:38.214216 6758 raft_consensus.cc:2454] T 895e37059dc7403d84b66d04cf3dc8fa P 68c15af52104439ca89aa126ebaa8661 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f80bb261c5594985aee14e2777537a5b in term 0.
I20260620 20:08:38.214376 6898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "895e37059dc7403d84b66d04cf3dc8fa" candidate_uuid: "f80bb261c5594985aee14e2777537a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e5e02c62e0f485491831f0ab22e0741" is_pre_election: true
I20260620 20:08:38.214534 6898 raft_consensus.cc:2454] T 895e37059dc7403d84b66d04cf3dc8fa P 0e5e02c62e0f485491831f0ab22e0741 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f80bb261c5594985aee14e2777537a5b in term 0.
I20260620 20:08:38.214762 6972 leader_election.cc:302] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c15af52104439ca89aa126ebaa8661, f80bb261c5594985aee14e2777537a5b; no voters:
I20260620 20:08:38.215008 7115 raft_consensus.cc:2781] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:08:38.215065 7115 raft_consensus.cc:482] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:08:38.215101 7115 raft_consensus.cc:3037] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:38.216034 7115 raft_consensus.cc:504] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } }
I20260620 20:08:38.216364 7115 leader_election.cc:288] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b [CANDIDATE]: Term 1 election: Requested vote from peers 68c15af52104439ca89aa126ebaa8661 (127.5.91.129:36633), 0e5e02c62e0f485491831f0ab22e0741 (127.5.91.130:40477)
I20260620 20:08:38.216641 6758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "895e37059dc7403d84b66d04cf3dc8fa" candidate_uuid: "f80bb261c5594985aee14e2777537a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c15af52104439ca89aa126ebaa8661"
I20260620 20:08:38.216758 6758 raft_consensus.cc:3037] T 895e37059dc7403d84b66d04cf3dc8fa P 68c15af52104439ca89aa126ebaa8661 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:38.216739 6898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "895e37059dc7403d84b66d04cf3dc8fa" candidate_uuid: "f80bb261c5594985aee14e2777537a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e5e02c62e0f485491831f0ab22e0741"
I20260620 20:08:38.216862 6898 raft_consensus.cc:3037] T 895e37059dc7403d84b66d04cf3dc8fa P 0e5e02c62e0f485491831f0ab22e0741 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:38.217760 6758 raft_consensus.cc:2454] T 895e37059dc7403d84b66d04cf3dc8fa P 68c15af52104439ca89aa126ebaa8661 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f80bb261c5594985aee14e2777537a5b in term 1.
I20260620 20:08:38.218039 6898 raft_consensus.cc:2454] T 895e37059dc7403d84b66d04cf3dc8fa P 0e5e02c62e0f485491831f0ab22e0741 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f80bb261c5594985aee14e2777537a5b in term 1.
I20260620 20:08:38.218075 6972 leader_election.cc:302] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c15af52104439ca89aa126ebaa8661, f80bb261c5594985aee14e2777537a5b; no voters:
I20260620 20:08:38.218341 7115 raft_consensus.cc:2781] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:38.218555 7115 raft_consensus.cc:686] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b [term 1 LEADER]: Becoming Leader. State: Replica: f80bb261c5594985aee14e2777537a5b, State: Running, Role: LEADER
I20260620 20:08:38.218792 7115 consensus_queue.cc:237] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } }
I20260620 20:08:38.220888 6600 catalog_manager.cc:5495] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b reported cstate change: term changed from 0 to 1, leader changed from <none> to f80bb261c5594985aee14e2777537a5b (127.5.91.131). New cstate: current_term: 1 leader_uuid: "f80bb261c5594985aee14e2777537a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } health_report { overall_health: HEALTHY } } }
Jun 20 20:08:38 dist-test-slave-l8h4 krb5kdc[6556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986117, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:38.281122 7144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:38.272074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:39281 (local address 127.0.0.1:51112)
0620 20:08:38.272387 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:38.272400 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:38.272512 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:38.272863 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:38.272865 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:38.273236 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:38.273333 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:38.273339 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.274460 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.274463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:38.275015 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:38.275023 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.275502 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.275505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:38.275580 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:38.276133 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:38.276145 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:38.277569 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:38.279412 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:38.279421 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:38.279427 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:38.279677 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:38.280029 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:38.280032 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:38.280033 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:38.280138 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:38.280624 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:38.280629 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:38.280735 (+ 106us) client_negotiation.cc:769] Sending connection context
0620 20:08:38.280858 (+ 123us) client_negotiation.cc:240] Negotiation successful
0620 20:08:38.280982 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":113,"threads_started":1}
I20260620 20:08:38.281160 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:38.272079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51112 (local address 127.5.91.190:39281)
0620 20:08:38.272230 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:38.272235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:38.272610 (+ 375us) server_negotiation.cc:407] Connection header received
0620 20:08:38.272658 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:38.272660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:38.272699 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:38.272782 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:38.273541 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.274302 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.275144 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.275354 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.275452 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:38.277753 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:38.277756 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:38.277758 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:38.277779 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:38.279279 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:38.279844 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:38.279847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:38.279848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:38.279924 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:38.280289 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:38.280292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:38.280293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:38.280494 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:38.280579 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:38.280952 (+ 373us) server_negotiation.cc:299] Negotiation successful
0620 20:08:38.281053 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260620 20:08:38.285094 6600 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:51112:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 20:08:38.285811 6600 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:08:38.290689 6737 tablet_service.cc:1467] Processing CreateTablet for tablet 690a804f8ee74989b770e9dd505dd192 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1c91e7f8ba05438eade854de5190f4b2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 20:08:38.290740 6878 tablet_service.cc:1467] Processing CreateTablet for tablet 690a804f8ee74989b770e9dd505dd192 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1c91e7f8ba05438eade854de5190f4b2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 20:08:38.291081 6737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 690a804f8ee74989b770e9dd505dd192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:38.291136 6878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 690a804f8ee74989b770e9dd505dd192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:38.291049 7018 tablet_service.cc:1467] Processing CreateTablet for tablet 690a804f8ee74989b770e9dd505dd192 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1c91e7f8ba05438eade854de5190f4b2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 20:08:38.291431 7018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 690a804f8ee74989b770e9dd505dd192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:38.293516 7107 tablet_bootstrap.cc:492] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661: Bootstrap starting.
I20260620 20:08:38.294059 7108 tablet_bootstrap.cc:492] T 690a804f8ee74989b770e9dd505dd192 P 0e5e02c62e0f485491831f0ab22e0741: Bootstrap starting.
I20260620 20:08:38.294380 7107 tablet_bootstrap.cc:654] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:38.294407 7111 tablet_bootstrap.cc:492] T 690a804f8ee74989b770e9dd505dd192 P f80bb261c5594985aee14e2777537a5b: Bootstrap starting.
I20260620 20:08:38.294966 7108 tablet_bootstrap.cc:654] T 690a804f8ee74989b770e9dd505dd192 P 0e5e02c62e0f485491831f0ab22e0741: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:38.295214 7111 tablet_bootstrap.cc:654] T 690a804f8ee74989b770e9dd505dd192 P f80bb261c5594985aee14e2777537a5b: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:38.295615 7107 tablet_bootstrap.cc:492] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661: No bootstrap required, opened a new log
I20260620 20:08:38.295691 7107 ts_tablet_manager.cc:1397] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260620 20:08:38.296010 7108 tablet_bootstrap.cc:492] T 690a804f8ee74989b770e9dd505dd192 P 0e5e02c62e0f485491831f0ab22e0741: No bootstrap required, opened a new log
I20260620 20:08:38.296084 7108 ts_tablet_manager.cc:1397] T 690a804f8ee74989b770e9dd505dd192 P 0e5e02c62e0f485491831f0ab22e0741: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 20:08:38.296058 7107 raft_consensus.cc:348] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } }
I20260620 20:08:38.296186 7107 raft_consensus.cc:374] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:38.296222 7107 raft_consensus.cc:729] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c15af52104439ca89aa126ebaa8661, State: Initialized, Role: FOLLOWER
I20260620 20:08:38.296343 7107 consensus_queue.cc:260] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } }
I20260620 20:08:38.296527 7108 raft_consensus.cc:348] T 690a804f8ee74989b770e9dd505dd192 P 0e5e02c62e0f485491831f0ab22e0741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } }
I20260620 20:08:38.296581 7107 ts_tablet_manager.cc:1428] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 20:08:38.296618 7108 raft_consensus.cc:374] T 690a804f8ee74989b770e9dd505dd192 P 0e5e02c62e0f485491831f0ab22e0741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:38.296640 7108 raft_consensus.cc:729] T 690a804f8ee74989b770e9dd505dd192 P 0e5e02c62e0f485491831f0ab22e0741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e5e02c62e0f485491831f0ab22e0741, State: Initialized, Role: FOLLOWER
I20260620 20:08:38.296739 7108 consensus_queue.cc:260] T 690a804f8ee74989b770e9dd505dd192 P 0e5e02c62e0f485491831f0ab22e0741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } }
I20260620 20:08:38.296766 7113 tablet_replica.cc:439] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } } }
I20260620 20:08:38.296900 7113 tablet_replica.cc:442] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 20:08:38.296986 7108 ts_tablet_manager.cc:1428] T 690a804f8ee74989b770e9dd505dd192 P 0e5e02c62e0f485491831f0ab22e0741: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 20:08:38.297444 7114 tablet_replica.cc:439] T 690a804f8ee74989b770e9dd505dd192 P 0e5e02c62e0f485491831f0ab22e0741: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } } }
I20260620 20:08:38.297556 7114 tablet_replica.cc:442] T 690a804f8ee74989b770e9dd505dd192 P 0e5e02c62e0f485491831f0ab22e0741: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 20:08:38.298722 7111 tablet_bootstrap.cc:492] T 690a804f8ee74989b770e9dd505dd192 P f80bb261c5594985aee14e2777537a5b: No bootstrap required, opened a new log
I20260620 20:08:38.298832 7111 ts_tablet_manager.cc:1397] T 690a804f8ee74989b770e9dd505dd192 P f80bb261c5594985aee14e2777537a5b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260620 20:08:38.299701 7111 raft_consensus.cc:348] T 690a804f8ee74989b770e9dd505dd192 P f80bb261c5594985aee14e2777537a5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } }
I20260620 20:08:38.299816 7111 raft_consensus.cc:374] T 690a804f8ee74989b770e9dd505dd192 P f80bb261c5594985aee14e2777537a5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:38.299841 7111 raft_consensus.cc:729] T 690a804f8ee74989b770e9dd505dd192 P f80bb261c5594985aee14e2777537a5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f80bb261c5594985aee14e2777537a5b, State: Initialized, Role: FOLLOWER
I20260620 20:08:38.299944 7111 consensus_queue.cc:260] T 690a804f8ee74989b770e9dd505dd192 P f80bb261c5594985aee14e2777537a5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } }
I20260620 20:08:38.300231 7111 ts_tablet_manager.cc:1428] T 690a804f8ee74989b770e9dd505dd192 P f80bb261c5594985aee14e2777537a5b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 20:08:38.300175 7115 tablet_replica.cc:439] T 690a804f8ee74989b770e9dd505dd192 P f80bb261c5594985aee14e2777537a5b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } } }
I20260620 20:08:38.300328 7115 tablet_replica.cc:442] T 690a804f8ee74989b770e9dd505dd192 P f80bb261c5594985aee14e2777537a5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 20:08:38.328684 7113 raft_consensus.cc:482] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:08:38.328843 7113 raft_consensus.cc:504] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } }
I20260620 20:08:38.329406 7113 leader_election.cc:288] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e5e02c62e0f485491831f0ab22e0741 (127.5.91.130:40477), f80bb261c5594985aee14e2777537a5b (127.5.91.131:36731)
I20260620 20:08:38.329766 6898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "690a804f8ee74989b770e9dd505dd192" candidate_uuid: "68c15af52104439ca89aa126ebaa8661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e5e02c62e0f485491831f0ab22e0741" is_pre_election: true
I20260620 20:08:38.329797 7038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "690a804f8ee74989b770e9dd505dd192" candidate_uuid: "68c15af52104439ca89aa126ebaa8661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f80bb261c5594985aee14e2777537a5b" is_pre_election: true
I20260620 20:08:38.329898 6898 raft_consensus.cc:2454] T 690a804f8ee74989b770e9dd505dd192 P 0e5e02c62e0f485491831f0ab22e0741 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68c15af52104439ca89aa126ebaa8661 in term 0.
I20260620 20:08:38.329941 7038 raft_consensus.cc:2454] T 690a804f8ee74989b770e9dd505dd192 P f80bb261c5594985aee14e2777537a5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68c15af52104439ca89aa126ebaa8661 in term 0.
I20260620 20:08:38.330200 6690 leader_election.cc:302] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e5e02c62e0f485491831f0ab22e0741, 68c15af52104439ca89aa126ebaa8661; no voters:
I20260620 20:08:38.330377 7113 raft_consensus.cc:2781] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:08:38.330435 7113 raft_consensus.cc:482] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:08:38.330479 7113 raft_consensus.cc:3037] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:38.331579 7113 raft_consensus.cc:504] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } }
I20260620 20:08:38.331939 7113 leader_election.cc:288] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661 [CANDIDATE]: Term 1 election: Requested vote from peers 0e5e02c62e0f485491831f0ab22e0741 (127.5.91.130:40477), f80bb261c5594985aee14e2777537a5b (127.5.91.131:36731)
I20260620 20:08:38.332489 6898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "690a804f8ee74989b770e9dd505dd192" candidate_uuid: "68c15af52104439ca89aa126ebaa8661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e5e02c62e0f485491831f0ab22e0741"
I20260620 20:08:38.332638 6898 raft_consensus.cc:3037] T 690a804f8ee74989b770e9dd505dd192 P 0e5e02c62e0f485491831f0ab22e0741 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:38.332653 7038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "690a804f8ee74989b770e9dd505dd192" candidate_uuid: "68c15af52104439ca89aa126ebaa8661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f80bb261c5594985aee14e2777537a5b"
I20260620 20:08:38.332826 7038 raft_consensus.cc:3037] T 690a804f8ee74989b770e9dd505dd192 P f80bb261c5594985aee14e2777537a5b [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:38.333642 6898 raft_consensus.cc:2454] T 690a804f8ee74989b770e9dd505dd192 P 0e5e02c62e0f485491831f0ab22e0741 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c15af52104439ca89aa126ebaa8661 in term 1.
I20260620 20:08:38.333694 7038 raft_consensus.cc:2454] T 690a804f8ee74989b770e9dd505dd192 P f80bb261c5594985aee14e2777537a5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c15af52104439ca89aa126ebaa8661 in term 1.
I20260620 20:08:38.333928 6690 leader_election.cc:302] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e5e02c62e0f485491831f0ab22e0741, 68c15af52104439ca89aa126ebaa8661; no voters:
I20260620 20:08:38.334089 7113 raft_consensus.cc:2781] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:38.334214 7113 raft_consensus.cc:686] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661 [term 1 LEADER]: Becoming Leader. State: Replica: 68c15af52104439ca89aa126ebaa8661, State: Running, Role: LEADER
I20260620 20:08:38.334343 7113 consensus_queue.cc:237] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } }
I20260620 20:08:38.335006 7118 tablet_replica.cc:439] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661: TxnStatusTablet state changed. Reason: New leader 68c15af52104439ca89aa126ebaa8661. Latest consensus state: current_term: 1 leader_uuid: "68c15af52104439ca89aa126ebaa8661" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } } }
I20260620 20:08:38.335088 7118 tablet_replica.cc:442] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661: This TxnStatusTablet replica's current role is: LEADER
I20260620 20:08:38.335407 7149 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 20:08:38.335709 7149 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 20:08:38.336189 6601 catalog_manager.cc:5495] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661 reported cstate change: term changed from 0 to 1, leader changed from <none> to 68c15af52104439ca89aa126ebaa8661 (127.5.91.129). New cstate: current_term: 1 leader_uuid: "68c15af52104439ca89aa126ebaa8661" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } health_report { overall_health: UNKNOWN } } }
I20260620 20:08:38.356083 7144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:38.352181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:36633 (local address 127.0.0.1:38442)
0620 20:08:38.352422 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:38.352439 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:38.352560 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:38.352831 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:38.352834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:38.352845 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 20:08:38.352957 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:38.352963 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.353834 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.353838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:38.354684 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:38.354694 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.355245 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.355249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:38.355339 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:38.355384 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 20:08:38.355693 (+ 309us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 20:08:38.355708 (+ 15us) client_negotiation.cc:769] Sending connection context
0620 20:08:38.355773 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 20:08:38.355860 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
I20260620 20:08:38.356083 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:38.352299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38442 (local address 127.5.91.129:36633)
0620 20:08:38.352431 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:38.352436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:38.352492 (+ 56us) server_negotiation.cc:407] Connection header received
0620 20:08:38.352678 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:38.352682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:38.352734 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:38.352808 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:08:38.353063 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.353710 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.354809 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.355114 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.355244 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:38.355539 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:08:38.355614 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:08:38.355682 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:38.355794 (+ 112us) server_negotiation.cc:299] Negotiation successful
0620 20:08:38.355849 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 20:08:38.358487 7038 raft_consensus.cc:1261] T 690a804f8ee74989b770e9dd505dd192 P f80bb261c5594985aee14e2777537a5b [term 1 FOLLOWER]: Refusing update from remote peer 68c15af52104439ca89aa126ebaa8661: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:08:38.358527 6898 raft_consensus.cc:1261] T 690a804f8ee74989b770e9dd505dd192 P 0e5e02c62e0f485491831f0ab22e0741 [term 1 FOLLOWER]: Refusing update from remote peer 68c15af52104439ca89aa126ebaa8661: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260620 20:08:38.358791 7085 tablet.cc:2369] T 690a804f8ee74989b770e9dd505dd192 P f80bb261c5594985aee14e2777537a5b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:08:38.359077 7113 consensus_queue.cc:1035] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:08:38.359226 7118 consensus_queue.cc:1035] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:08:38.360469 7114 tablet_replica.cc:439] T 690a804f8ee74989b770e9dd505dd192 P 0e5e02c62e0f485491831f0ab22e0741: TxnStatusTablet state changed. Reason: New leader 68c15af52104439ca89aa126ebaa8661. Latest consensus state: current_term: 1 leader_uuid: "68c15af52104439ca89aa126ebaa8661" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } } }
I20260620 20:08:38.360574 7114 tablet_replica.cc:442] T 690a804f8ee74989b770e9dd505dd192 P 0e5e02c62e0f485491831f0ab22e0741: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 20:08:38.360538 7115 tablet_replica.cc:439] T 690a804f8ee74989b770e9dd505dd192 P f80bb261c5594985aee14e2777537a5b: TxnStatusTablet state changed. Reason: New leader 68c15af52104439ca89aa126ebaa8661. Latest consensus state: current_term: 1 leader_uuid: "68c15af52104439ca89aa126ebaa8661" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } } }
I20260620 20:08:38.360644 7115 tablet_replica.cc:442] T 690a804f8ee74989b770e9dd505dd192 P f80bb261c5594985aee14e2777537a5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 20:08:38.362124 7113 tablet_replica.cc:439] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "68c15af52104439ca89aa126ebaa8661" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } } }
I20260620 20:08:38.362275 7113 tablet_replica.cc:442] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661: This TxnStatusTablet replica's current role is: LEADER
I20260620 20:08:38.363529 7114 tablet_replica.cc:439] T 690a804f8ee74989b770e9dd505dd192 P 0e5e02c62e0f485491831f0ab22e0741: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "68c15af52104439ca89aa126ebaa8661" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } } }
I20260620 20:08:38.363636 7114 tablet_replica.cc:442] T 690a804f8ee74989b770e9dd505dd192 P 0e5e02c62e0f485491831f0ab22e0741: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 20:08:38.366339 7118 tablet_replica.cc:439] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "68c15af52104439ca89aa126ebaa8661" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } } }
I20260620 20:08:38.366442 7118 tablet_replica.cc:442] T 690a804f8ee74989b770e9dd505dd192 P 68c15af52104439ca89aa126ebaa8661: This TxnStatusTablet replica's current role is: LEADER
I20260620 20:08:38.366943 7122 mvcc.cc:205] Tried to move back new op lower bound from 7299015140792557568 to 7299015140698652672. Current Snapshot: MvccSnapshot[applied={T|T < 7299015140792557568}]
I20260620 20:08:38.367096 7115 tablet_replica.cc:439] T 690a804f8ee74989b770e9dd505dd192 P f80bb261c5594985aee14e2777537a5b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "68c15af52104439ca89aa126ebaa8661" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 } } peers { permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 } } peers { permanent_uuid: "f80bb261c5594985aee14e2777537a5b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 36731 } } }
I20260620 20:08:38.367183 7115 tablet_replica.cc:442] T 690a804f8ee74989b770e9dd505dd192 P f80bb261c5594985aee14e2777537a5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 20:08:38.371706 7106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:38.367778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40194 (local address 127.5.91.131:36731)
0620 20:08:38.367872 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:38.367875 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:38.367885 (+ 10us) server_negotiation.cc:407] Connection header received
0620 20:08:38.367952 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:38.367956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:38.368014 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:38.368113 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:08:38.368581 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.369167 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.370311 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.370546 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.370628 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:38.370905 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:08:38.371363 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:08:38.371470 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:38.371562 (+ 92us) server_negotiation.cc:299] Negotiation successful
0620 20:08:38.371608 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 20:08:38.381748 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:38.378215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:58691 (local address 127.5.91.129:36633)
0620 20:08:38.378343 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:38.378347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:38.378498 (+ 151us) server_negotiation.cc:407] Connection header received
0620 20:08:38.378602 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:38.378604 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:38.378655 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:38.378722 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:08:38.378968 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.379461 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.380613 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.380950 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.381053 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:38.381376 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:08:38.381449 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:08:38.381506 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:38.381593 (+ 87us) server_negotiation.cc:299] Negotiation successful
0620 20:08:38.381640 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260620 20:08:38.381816 7155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:38.378072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:36633 (local address 127.5.91.131:58691)
0620 20:08:38.378372 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:38.378387 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:38.378502 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:38.378738 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:38.378741 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:38.378749 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 20:08:38.378855 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:38.378861 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.379546 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.379549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:38.380479 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:38.380490 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.381081 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.381085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:38.381187 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:38.381239 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 20:08:38.381519 (+ 280us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 20:08:38.381524 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 20:08:38.381585 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 20:08:38.381662 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":124,"threads_started":1}
I20260620 20:08:38.389001 6898 raft_consensus.cc:1261] T 895e37059dc7403d84b66d04cf3dc8fa P 0e5e02c62e0f485491831f0ab22e0741 [term 1 FOLLOWER]: Refusing update from remote peer f80bb261c5594985aee14e2777537a5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:08:38.389253 6758 raft_consensus.cc:1261] T 895e37059dc7403d84b66d04cf3dc8fa P 68c15af52104439ca89aa126ebaa8661 [term 1 FOLLOWER]: Refusing update from remote peer f80bb261c5594985aee14e2777537a5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:08:38.389834 7115 consensus_queue.cc:1035] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "68c15af52104439ca89aa126ebaa8661" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 36633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:08:38.390066 7137 consensus_queue.cc:1035] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e5e02c62e0f485491831f0ab22e0741" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:08:38.404574 7123 mvcc.cc:205] Tried to move back new op lower bound from 7299015140917366784 to 7299015140226179072. Current Snapshot: MvccSnapshot[applied={T|T < 7299015140917366784}]
I20260620 20:08:38.416561 7106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:38.411843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:49941 (local address 127.5.91.131:36731)
0620 20:08:38.411947 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:38.411951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:38.412243 (+ 292us) server_negotiation.cc:407] Connection header received
0620 20:08:38.412379 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:38.412382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:38.412434 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:38.412500 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:08:38.412730 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.413508 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.415174 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.415620 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.415701 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:38.416094 (+ 393us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:08:38.416178 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:08:38.416232 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:38.416377 (+ 145us) server_negotiation.cc:299] Negotiation successful
0620 20:08:38.416433 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 20:08:38.417173 7160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:38.411502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:36731 (local address 127.5.91.129:49941)
0620 20:08:38.412166 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:38.412180 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:38.412283 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:38.412519 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:38.412522 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:38.412529 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 20:08:38.412638 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:38.412643 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.413611 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.413614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:38.415024 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:38.415036 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.415819 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.415823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:38.415909 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:38.415959 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 20:08:38.416289 (+ 330us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 20:08:38.416294 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 20:08:38.416944 (+ 650us) client_negotiation.cc:240] Negotiation successful
0620 20:08:38.417033 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":114,"threads_started":1}
I20260620 20:08:38.449019 6601 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:08:38.449209 6601 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:08:38.451575 6601 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 87ced1148c5149c996424e16bbafd4f6: Sending DeleteTablet for 3 replicas of tablet 895e37059dc7403d84b66d04cf3dc8fa
I20260620 20:08:38.452441 6878 tablet_service.cc:1514] Processing DeleteTablet for tablet 895e37059dc7403d84b66d04cf3dc8fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:38 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:37450
I20260620 20:08:38.452639 7018 tablet_service.cc:1514] Processing DeleteTablet for tablet 895e37059dc7403d84b66d04cf3dc8fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:38 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:40186
I20260620 20:08:38.452821 7131 tablet_replica.cc:330] T 895e37059dc7403d84b66d04cf3dc8fa P 0e5e02c62e0f485491831f0ab22e0741: stopping tablet replica
I20260620 20:08:38.452934 7131 raft_consensus.cc:2229] T 895e37059dc7403d84b66d04cf3dc8fa P 0e5e02c62e0f485491831f0ab22e0741 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:08:38.453043 7131 raft_consensus.cc:2258] T 895e37059dc7403d84b66d04cf3dc8fa P 0e5e02c62e0f485491831f0ab22e0741 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:08:38.453178 7132 tablet_replica.cc:330] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b: stopping tablet replica
I20260620 20:08:38.453284 7132 raft_consensus.cc:2229] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b [term 1 LEADER]: Raft consensus shutting down.
I20260620 20:08:38.453560 7132 raft_consensus.cc:2258] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:08:38.454465 7132 ts_tablet_manager.cc:1905] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:08:38.454569 7131 ts_tablet_manager.cc:1905] T 895e37059dc7403d84b66d04cf3dc8fa P 0e5e02c62e0f485491831f0ab22e0741: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:08:38.457289 7132 ts_tablet_manager.cc:1918] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 20:08:38.457300 7131 ts_tablet_manager.cc:1918] T 895e37059dc7403d84b66d04cf3dc8fa P 0e5e02c62e0f485491831f0ab22e0741: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 20:08:38.457355 7132 log.cc:1196] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/895e37059dc7403d84b66d04cf3dc8fa
I20260620 20:08:38.457367 7131 log.cc:1196] T 895e37059dc7403d84b66d04cf3dc8fa P 0e5e02c62e0f485491831f0ab22e0741: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-1/wal/wals/895e37059dc7403d84b66d04cf3dc8fa
I20260620 20:08:38.457592 7132 ts_tablet_manager.cc:1939] T 895e37059dc7403d84b66d04cf3dc8fa P f80bb261c5594985aee14e2777537a5b: Deleting consensus metadata
I20260620 20:08:38.457608 7131 ts_tablet_manager.cc:1939] T 895e37059dc7403d84b66d04cf3dc8fa P 0e5e02c62e0f485491831f0ab22e0741: Deleting consensus metadata
I20260620 20:08:38.457919 6737 tablet_service.cc:1514] Processing DeleteTablet for tablet 895e37059dc7403d84b66d04cf3dc8fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:38 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:38416
I20260620 20:08:38.458105 7130 tablet_replica.cc:330] T 895e37059dc7403d84b66d04cf3dc8fa P 68c15af52104439ca89aa126ebaa8661: stopping tablet replica
I20260620 20:08:38.458242 7130 raft_consensus.cc:2229] T 895e37059dc7403d84b66d04cf3dc8fa P 68c15af52104439ca89aa126ebaa8661 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:08:38.458364 7130 raft_consensus.cc:2258] T 895e37059dc7403d84b66d04cf3dc8fa P 68c15af52104439ca89aa126ebaa8661 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:08:38.459112 7130 ts_tablet_manager.cc:1905] T 895e37059dc7403d84b66d04cf3dc8fa P 68c15af52104439ca89aa126ebaa8661: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:08:38.459115 6586 catalog_manager.cc:4841] TS 0e5e02c62e0f485491831f0ab22e0741 (127.5.91.130:40477): tablet 895e37059dc7403d84b66d04cf3dc8fa (table test-table [id=622f273861ce499a9ca087bd1f7f609a]) successfully deleted
I20260620 20:08:38.459296 6588 catalog_manager.cc:4841] TS f80bb261c5594985aee14e2777537a5b (127.5.91.131:36731): tablet 895e37059dc7403d84b66d04cf3dc8fa (table test-table [id=622f273861ce499a9ca087bd1f7f609a]) successfully deleted
I20260620 20:08:38.461531 7130 ts_tablet_manager.cc:1918] T 895e37059dc7403d84b66d04cf3dc8fa P 68c15af52104439ca89aa126ebaa8661: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 20:08:38.461601 7130 log.cc:1196] T 895e37059dc7403d84b66d04cf3dc8fa P 68c15af52104439ca89aa126ebaa8661: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781986111135528-5486-0/minicluster-data/ts-0/wal/wals/895e37059dc7403d84b66d04cf3dc8fa
I20260620 20:08:38.461820 7130 ts_tablet_manager.cc:1939] T 895e37059dc7403d84b66d04cf3dc8fa P 68c15af52104439ca89aa126ebaa8661: Deleting consensus metadata
I20260620 20:08:38.462545 6588 catalog_manager.cc:4841] TS 68c15af52104439ca89aa126ebaa8661 (127.5.91.129:36633): tablet 895e37059dc7403d84b66d04cf3dc8fa (table test-table [id=622f273861ce499a9ca087bd1f7f609a]) successfully deleted
Jun 20 20:08:38 dist-test-slave-l8h4 krb5kdc[6556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986117, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.91.129@KRBTEST.COM
I20260620 20:08:38.471284 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:38.458489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38452 (local address 127.5.91.129:36633)
0620 20:08:38.458610 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:38.458614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:38.458968 (+ 354us) server_negotiation.cc:407] Connection header received
0620 20:08:38.459480 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:38.459483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:38.459535 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:38.459595 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:38.460592 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.461347 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.462702 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.463039 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.463119 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:38.466997 (+ 3878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:38.467010 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:38.467028 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:38.467056 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:38.469090 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:38.469618 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:38.469625 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:38.469627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:38.469736 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:38.470057 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:38.470060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:38.470062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:38.470412 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:38.470592 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:38.470980 (+ 388us) server_negotiation.cc:299] Negotiation successful
0620 20:08:38.471107 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260620 20:08:38.472383 6798 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38452
I20260620 20:08:38.483527 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:38.474192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51118 (local address 127.5.91.190:39281)
0620 20:08:38.474314 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:38.474318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:38.474725 (+ 407us) server_negotiation.cc:407] Connection header received
0620 20:08:38.474851 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:38.474855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:38.474906 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:38.474981 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:38.476243 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.477062 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.478192 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:38.478550 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:38.478650 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:38.480108 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:38.480112 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:38.480115 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:38.480142 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:38.481725 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:38.482237 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:38.482241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:38.482243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:38.482358 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:38.482616 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:38.482619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:38.482621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:38.482837 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:38.482963 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:38.483222 (+ 259us) server_negotiation.cc:299] Negotiation successful
0620 20:08:38.483350 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260620 20:08:38.484093 6601 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51118
I20260620 20:08:38.485440 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 6673
I20260620 20:08:38.492337 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 6813
I20260620 20:08:38.499675 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 6953
I20260620 20:08:38.504894 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 6569
2026-06-20T20:08:38Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2756 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:08:38 dist-test-slave-l8h4 krb5kdc[7169](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:08:38 dist-test-slave-l8h4 krb5kdc[7169](info): set up 2 sockets
Jun 20 20:08:38 dist-test-slave-l8h4 krb5kdc[7169](info): commencing operation
krb5kdc: starting...
W20260620 20:08:39.487800 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.961s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:08:39 dist-test-slave-l8h4 krb5kdc[7169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:08:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:08:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:08:39.615281 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:40343
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:40343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:39.744717 7182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:39.745021 7182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:39.745064 7182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:39.749078 7182 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:08:39.749150 7182 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 20:08:39.749168 7182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:39.749189 7182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:39.749209 7182 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:08:39.749228 7182 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:08:39.756116 7182 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:40343
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:40343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:39.757185 7182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:39.758193 7182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:39.767505 7190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:39.767511 7187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:39.767764 7182 server_base.cc:1034] running on GCE node
W20260620 20:08:39.767505 7188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:39.768451 7182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:39.769587 7182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:39.770835 7182 hybrid_clock.cc:648] HybridClock initialized: now 1781986119770775 us; error 64 us; skew 500 ppm
Jun 20 20:08:39 dist-test-slave-l8h4 krb5kdc[7169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986119, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:39.776046 7182 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:08:39.777361 7182 webserver.cc:460] Webserver started at http://127.5.91.190:43209/ using document root <none> and password file <none>
I20260620 20:08:39.778367 7182 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:39.778508 7182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:39.779184 7182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:39.782449 7182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "f7d11bf80d7548788bed233a0d778122"
format_stamp: "Formatted at 2026-06-20 20:08:39 on dist-test-slave-l8h4"
server_key: "150e16369df6c799960c1b64815d2b23"
server_key_iv: "386ddd18b85b91557855d5d232ccd7a9"
server_key_version: "encryptionkey@0"
I20260620 20:08:39.783185 7182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "f7d11bf80d7548788bed233a0d778122"
format_stamp: "Formatted at 2026-06-20 20:08:39 on dist-test-slave-l8h4"
server_key: "150e16369df6c799960c1b64815d2b23"
server_key_iv: "386ddd18b85b91557855d5d232ccd7a9"
server_key_version: "encryptionkey@0"
I20260620 20:08:39.788970 7182 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 20:08:39.792073 7197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:39.793498 7182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 20:08:39.793629 7182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "f7d11bf80d7548788bed233a0d778122"
format_stamp: "Formatted at 2026-06-20 20:08:39 on dist-test-slave-l8h4"
server_key: "150e16369df6c799960c1b64815d2b23"
server_key_iv: "386ddd18b85b91557855d5d232ccd7a9"
server_key_version: "encryptionkey@0"
I20260620 20:08:39.793754 7182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:39.807138 7182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:39.810966 7182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:39.811138 7182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:39.821321 7182 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:40343
I20260620 20:08:39.821456 7259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:40343 every 8 connection(s)
I20260620 20:08:39.822507 7182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:08:39.824128 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 7182
I20260620 20:08:39.824290 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:08:39.824651 5486 external_mini_cluster.cc:1428] Setting key 3f243c1cb7dcedb3bc26314eab770109
I20260620 20:08:39.827351 7260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:39.834229 7260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122: Bootstrap starting.
Jun 20 20:08:39 dist-test-slave-l8h4 krb5kdc[7169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:39.836859 7260 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:39.838084 7260 log.cc:824] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:39.841430 7260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122: No bootstrap required, opened a new log
I20260620 20:08:39.842871 7263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:39.826756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51278 (local address 127.5.91.190:40343)
0620 20:08:39.827427 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:39.827454 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:39.827586 (+ 132us) server_negotiation.cc:407] Connection header received
0620 20:08:39.828780 (+ 1194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:39.828810 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:39.829291 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:39.829880 (+ 589us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:39.830765 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:39.831804 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:39.832639 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:39.833121 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:39.833264 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:39.836212 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:39.836218 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:39.836237 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:39.836272 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:39.838562 (+ 2290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:39.839279 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:39.839289 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:39.839299 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:39.839544 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:39.840079 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:39.840083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:39.840085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:39.840909 (+ 824us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:39.841141 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:39.841499 (+ 358us) server_negotiation.cc:299] Negotiation successful
0620 20:08:39.841834 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":178,"threads_started":1}
I20260620 20:08:39.845203 7260 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7d11bf80d7548788bed233a0d778122" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40343 } }
I20260620 20:08:39.845453 7260 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:39.845491 7260 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7d11bf80d7548788bed233a0d778122, State: Initialized, Role: FOLLOWER
I20260620 20:08:39.845875 7260 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7d11bf80d7548788bed233a0d778122" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40343 } }
I20260620 20:08:39.846007 7260 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:08:39.846047 7260 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:08:39.846110 7260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:39.847064 7260 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7d11bf80d7548788bed233a0d778122" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40343 } }
I20260620 20:08:39.847358 7260 leader_election.cc:302] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7d11bf80d7548788bed233a0d778122; no voters:
I20260620 20:08:39.847652 7260 leader_election.cc:288] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:08:39.847782 7265 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:39.847994 7265 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122 [term 1 LEADER]: Becoming Leader. State: Replica: f7d11bf80d7548788bed233a0d778122, State: Running, Role: LEADER
I20260620 20:08:39.848286 7265 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7d11bf80d7548788bed233a0d778122" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40343 } }
I20260620 20:08:39.848809 7260 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:08:39.850075 7266 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7d11bf80d7548788bed233a0d778122" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7d11bf80d7548788bed233a0d778122" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40343 } } }
I20260620 20:08:39.850236 7266 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122 [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:39.850916 7267 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7d11bf80d7548788bed233a0d778122. Latest consensus state: current_term: 1 leader_uuid: "f7d11bf80d7548788bed233a0d778122" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7d11bf80d7548788bed233a0d778122" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40343 } } }
I20260620 20:08:39.851033 7267 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122 [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:39.852109 7276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:08:39.855057 7276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:08:39.861007 7276 catalog_manager.cc:1347] Generated new cluster ID: c399dab20d814110b8e7cfffaa63f81a
I20260620 20:08:39.861083 7276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:08:39.892622 7276 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:08:39.894237 7276 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:08:39.913153 7276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122: Generated new TSK 0
I20260620 20:08:39.914108 7276 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.91.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
I20260620 20:08:39.949126 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:40343
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:40.070070 7286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:40.070359 7286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:40.070408 7286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:40.074496 7286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:40.074561 7286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:40.074602 7286 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 20:08:40.074615 7286 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 20:08:40.074653 7286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:08:40.080298 7286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.91.190:40343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:40.081434 7286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:40.082453 7286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:40.092021 7291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:40.092007 7292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:40.091981 7294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:40.092056 7286 server_base.cc:1034] running on GCE node
I20260620 20:08:40.092670 7286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:40.093333 7286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:40.094645 7286 hybrid_clock.cc:648] HybridClock initialized: now 1781986120094586 us; error 52 us; skew 500 ppm
Jun 20 20:08:40 dist-test-slave-l8h4 krb5kdc[7169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986120, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:40.097646 7286 init.cc:377] Logged in from keytab as kudu/127.5.91.129@KRBTEST.COM (short username kudu)
I20260620 20:08:40.098742 7286 webserver.cc:460] Webserver started at http://127.5.91.129:32885/ using document root <none> and password file <none>
I20260620 20:08:40.099324 7286 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:40.099391 7286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:40.099596 7286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:40.101812 7286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "f390f74c2612442c903fc4ddca73be71"
format_stamp: "Formatted at 2026-06-20 20:08:40 on dist-test-slave-l8h4"
server_key: "d23f37a568630d1082882d3c81f1c746"
server_key_iv: "30f40856e80a565ebff980931d47ae7a"
server_key_version: "encryptionkey@0"
I20260620 20:08:40.102417 7286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "f390f74c2612442c903fc4ddca73be71"
format_stamp: "Formatted at 2026-06-20 20:08:40 on dist-test-slave-l8h4"
server_key: "d23f37a568630d1082882d3c81f1c746"
server_key_iv: "30f40856e80a565ebff980931d47ae7a"
server_key_version: "encryptionkey@0"
I20260620 20:08:40.107916 7286 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260620 20:08:40.111276 7301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:40.112874 7286 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 20:08:40.113025 7286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "f390f74c2612442c903fc4ddca73be71"
format_stamp: "Formatted at 2026-06-20 20:08:40 on dist-test-slave-l8h4"
server_key: "d23f37a568630d1082882d3c81f1c746"
server_key_iv: "30f40856e80a565ebff980931d47ae7a"
server_key_version: "encryptionkey@0"
I20260620 20:08:40.113158 7286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:40.139101 7286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:40.142308 7286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:40.142498 7286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:40.143910 7286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:40.143960 7286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:40.144013 7286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:40.144042 7286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:40.154521 7286 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:40759
I20260620 20:08:40.154614 7416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:40759 every 8 connection(s)
Jun 20 20:08:40 dist-test-slave-l8h4 krb5kdc[7169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986120, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:40.155437 7286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:08:40.157085 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 7286
I20260620 20:08:40.157200 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:08:40.157497 5486 external_mini_cluster.cc:1428] Setting key f8151d8f4249273aa8a20716abdbed6c
I20260620 20:08:40.160087 7263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.144792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:50277 (local address 127.5.91.190:40343)
0620 20:08:40.144923 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:40.144928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:40.146445 (+ 1517us) server_negotiation.cc:407] Connection header received
0620 20:08:40.147975 (+ 1530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:40.147978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:40.148043 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:40.148149 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:40.149487 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.150322 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.152009 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.152265 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.152363 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.155284 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:40.155288 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:40.155291 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:40.155325 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:40.157423 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:40.158034 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:40.158037 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:40.158040 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:40.158133 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:40.158591 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:40.158596 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:40.158598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:40.158964 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:40.159134 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:40.159741 (+ 607us) server_negotiation.cc:299] Negotiation successful
0620 20:08:40.159921 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 20:08:40.160931 7310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.145328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:40343 (local address 127.5.91.129:50277)
0620 20:08:40.146127 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:40.146273 (+ 146us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:40.147709 (+ 1436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:40.148414 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:40.148423 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:40.148764 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:40.149287 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.149307 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.150491 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.150495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.151890 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.151901 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.152440 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.152445 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.152651 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.153335 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:40.153367 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:40.155117 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:40.157546 (+ 2429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:40.157558 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:40.157573 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:40.157916 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:40.158257 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:40.158261 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:40.158263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:40.158405 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:40.159160 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:40.159167 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:40.159487 (+ 320us) client_negotiation.cc:769] Sending connection context
0620 20:08:40.159670 (+ 183us) client_negotiation.cc:240] Negotiation successful
0620 20:08:40.160077 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":135,"threads_started":1}
I20260620 20:08:40.162524 7417 heartbeater.cc:347] Connected to a master server at 127.5.91.190:40343
I20260620 20:08:40.162671 7417 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:40.163161 7417 heartbeater.cc:511] Master 127.5.91.190:40343 requested a full tablet report, sending...
I20260620 20:08:40.164711 7213 ts_manager.cc:194] Registered new tserver with Master: f390f74c2612442c903fc4ddca73be71 (127.5.91.129:40759)
I20260620 20:08:40.166250 7213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:50277
WARNING: no policy specified for kudu/127.5.91.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.130@KRBTEST.COM" created.
I20260620 20:08:40.173216 7424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.164538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:40343 (local address 127.5.91.129:34155)
0620 20:08:40.164827 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:40.164840 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:40.164960 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:40.165180 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:40.165183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:40.165387 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:40.165481 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.165487 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.166225 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.166228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.166856 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.166865 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.167463 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.167466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.167599 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.168146 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:40.168161 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:40.168582 (+ 421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:40.171097 (+ 2515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:40.171100 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:40.171101 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:40.171302 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:40.171752 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:40.171756 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:40.171758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:40.171915 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:40.172585 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:40.172589 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:40.172729 (+ 140us) client_negotiation.cc:769] Sending connection context
0620 20:08:40.172909 (+ 180us) client_negotiation.cc:240] Negotiation successful
0620 20:08:40.173057 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":3,"thread_start_us":78,"threads_started":1}
I20260620 20:08:40.173488 7263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.164608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:34155 (local address 127.5.91.190:40343)
0620 20:08:40.164720 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:40.164724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:40.164898 (+ 174us) server_negotiation.cc:407] Connection header received
0620 20:08:40.165047 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:40.165050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:40.165091 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:40.165154 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:40.165583 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.166094 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.166984 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.167232 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.167314 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.168740 (+ 1426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:40.168744 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:40.168747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:40.168771 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:40.170963 (+ 2192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:40.171398 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:40.171401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:40.171402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:40.171493 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:40.172116 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:40.172120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:40.172122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:40.172388 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:40.172590 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:40.172998 (+ 408us) server_negotiation.cc:299] Negotiation successful
0620 20:08:40.173144 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
I20260620 20:08:40.191123 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:40343
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:40.307458 7426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:40.307750 7426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:40.307821 7426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:40.312976 7426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:40.313063 7426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:40.313124 7426 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 20:08:40.313146 7426 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 20:08:40.313201 7426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:08:40.320319 7426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.91.190:40343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:40.321448 7426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:40.322486 7426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:40.332334 7431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:40.332369 7432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:40.332581 7434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:40.332435 7426 server_base.cc:1034] running on GCE node
I20260620 20:08:40.332986 7426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:40.333618 7426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:40.334852 7426 hybrid_clock.cc:648] HybridClock initialized: now 1781986120334796 us; error 53 us; skew 500 ppm
Jun 20 20:08:40 dist-test-slave-l8h4 krb5kdc[7169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986120, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:40.338028 7426 init.cc:377] Logged in from keytab as kudu/127.5.91.130@KRBTEST.COM (short username kudu)
I20260620 20:08:40.339605 7426 webserver.cc:460] Webserver started at http://127.5.91.130:41233/ using document root <none> and password file <none>
I20260620 20:08:40.340267 7426 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:40.340341 7426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:40.340873 7426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:40.343393 7426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "16b0d7dcc332467fb0bfad2531110705"
format_stamp: "Formatted at 2026-06-20 20:08:40 on dist-test-slave-l8h4"
server_key: "2af5ed4342c03ffa6e661124edd18d5a"
server_key_iv: "d2b2f20955a78e2d2be9836ddee0b744"
server_key_version: "encryptionkey@0"
I20260620 20:08:40.344079 7426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "16b0d7dcc332467fb0bfad2531110705"
format_stamp: "Formatted at 2026-06-20 20:08:40 on dist-test-slave-l8h4"
server_key: "2af5ed4342c03ffa6e661124edd18d5a"
server_key_iv: "d2b2f20955a78e2d2be9836ddee0b744"
server_key_version: "encryptionkey@0"
I20260620 20:08:40.348364 7426 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260620 20:08:40.351589 7441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:40.353472 7426 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260620 20:08:40.353586 7426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "16b0d7dcc332467fb0bfad2531110705"
format_stamp: "Formatted at 2026-06-20 20:08:40 on dist-test-slave-l8h4"
server_key: "2af5ed4342c03ffa6e661124edd18d5a"
server_key_iv: "d2b2f20955a78e2d2be9836ddee0b744"
server_key_version: "encryptionkey@0"
I20260620 20:08:40.353688 7426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:40.366541 7426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:40.370596 7426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:40.370764 7426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:40.372243 7426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:40.372300 7426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:40.372339 7426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:40.372352 7426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 20:08:40 dist-test-slave-l8h4 krb5kdc[7169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986120, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:40.382359 7426 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:41225
I20260620 20:08:40.382401 7556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:41225 every 8 connection(s)
I20260620 20:08:40.383374 7426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:08:40.386310 7263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.373146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:34695 (local address 127.5.91.190:40343)
0620 20:08:40.373267 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:40.373271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:40.374248 (+ 977us) server_negotiation.cc:407] Connection header received
0620 20:08:40.375228 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:40.375232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:40.375285 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:40.375395 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:40.376630 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.377441 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.378851 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.379065 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.379152 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.382351 (+ 3199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:40.382355 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:40.382357 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:40.382382 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:40.383773 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:40.384557 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:40.384560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:40.384562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:40.384652 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:40.384997 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:40.384999 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:40.385000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:40.385176 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:40.385252 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:40.386025 (+ 773us) server_negotiation.cc:299] Negotiation successful
0620 20:08:40.386132 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 20:08:40.386999 7450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.373601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:40343 (local address 127.5.91.130:34695)
0620 20:08:40.374077 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:40.374117 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:40.375021 (+ 904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:40.375606 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:40.375613 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:40.375971 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:40.376494 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.376508 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.377569 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.377573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.378741 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.378751 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.379214 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.379218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.379389 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.380544 (+ 1155us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:40.380566 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:40.382046 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:40.383931 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:40.383940 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:40.383963 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:40.384420 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:40.384748 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:40.384751 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:40.384752 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:40.384881 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:40.385251 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:40.385256 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:40.385724 (+ 468us) client_negotiation.cc:769] Sending connection context
0620 20:08:40.385923 (+ 199us) client_negotiation.cc:240] Negotiation successful
0620 20:08:40.386211 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":127,"threads_started":1}
I20260620 20:08:40.388214 7557 heartbeater.cc:347] Connected to a master server at 127.5.91.190:40343
I20260620 20:08:40.388370 7557 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:40.388406 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 7426
I20260620 20:08:40.388516 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:08:40.388811 5486 external_mini_cluster.cc:1428] Setting key 00dfc76968ea15d0444c3b0ec7fba770
I20260620 20:08:40.388957 7557 heartbeater.cc:511] Master 127.5.91.190:40343 requested a full tablet report, sending...
I20260620 20:08:40.389887 7213 ts_manager.cc:194] Registered new tserver with Master: 16b0d7dcc332467fb0bfad2531110705 (127.5.91.130:41225)
I20260620 20:08:40.390607 7213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:34695
I20260620 20:08:40.397827 7263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.390399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:35015 (local address 127.5.91.190:40343)
0620 20:08:40.390506 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:40.390510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:40.390731 (+ 221us) server_negotiation.cc:407] Connection header received
0620 20:08:40.390933 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:40.390937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:40.390985 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:40.391075 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:40.391463 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.392066 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.393153 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.393389 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.393483 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.394656 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:40.394659 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:40.394661 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:40.394682 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:40.396049 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:40.396526 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:40.396529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:40.396530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:40.396607 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:40.396880 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:40.396883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:40.396884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:40.397089 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:40.397165 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:40.397585 (+ 420us) server_negotiation.cc:299] Negotiation successful
0620 20:08:40.397687 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 20:08:40.397825 7563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.390437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:40343 (local address 127.5.91.130:35015)
0620 20:08:40.390658 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:40.390672 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:40.390790 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:40.391097 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:40.391099 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:40.391265 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:40.391357 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.391363 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.392192 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.392195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.393018 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.393027 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.393511 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.393514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.393597 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.394082 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:40.394096 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:40.394521 (+ 425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:40.396159 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:40.396162 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:40.396164 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:40.396415 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:40.396695 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:40.396698 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:40.396700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:40.396790 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:40.397190 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:40.397193 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:40.397322 (+ 129us) client_negotiation.cc:769] Sending connection context
0620 20:08:40.397470 (+ 148us) client_negotiation.cc:240] Negotiation successful
0620 20:08:40.397616 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":86,"threads_started":1}
WARNING: no policy specified for kudu/127.5.91.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
I20260620 20:08:40.418586 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:40343
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:40.530983 7566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:40.531265 7566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:40.531329 7566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:40.535715 7566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:40.535784 7566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:40.535840 7566 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 20:08:40.535868 7566 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 20:08:40.535933 7566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:08:40.542642 7566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.91.190:40343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:40.543795 7566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:40.544845 7566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:40.554620 7574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:40.554719 7566 server_base.cc:1034] running on GCE node
W20260620 20:08:40.554643 7572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:40.554643 7571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:40.555197 7566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:40.555799 7566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:40.557000 7566 hybrid_clock.cc:648] HybridClock initialized: now 1781986120556977 us; error 32 us; skew 500 ppm
Jun 20 20:08:40 dist-test-slave-l8h4 krb5kdc[7169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986120, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:40.564101 7566 init.cc:377] Logged in from keytab as kudu/127.5.91.131@KRBTEST.COM (short username kudu)
I20260620 20:08:40.565513 7566 webserver.cc:460] Webserver started at http://127.5.91.131:33451/ using document root <none> and password file <none>
I20260620 20:08:40.566076 7566 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:40.566172 7566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:40.566375 7566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:40.568392 7566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "2e5414be5a3c427595e36ef50951dc46"
format_stamp: "Formatted at 2026-06-20 20:08:40 on dist-test-slave-l8h4"
server_key: "810b863d7e37d6adba443cd848f74d7c"
server_key_iv: "05abd20c9fd890c4c455592d827e09ca"
server_key_version: "encryptionkey@0"
I20260620 20:08:40.568921 7566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "2e5414be5a3c427595e36ef50951dc46"
format_stamp: "Formatted at 2026-06-20 20:08:40 on dist-test-slave-l8h4"
server_key: "810b863d7e37d6adba443cd848f74d7c"
server_key_iv: "05abd20c9fd890c4c455592d827e09ca"
server_key_version: "encryptionkey@0"
I20260620 20:08:40.572644 7566 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 20:08:40.575026 7581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:40.576285 7566 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:08:40.576391 7566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "2e5414be5a3c427595e36ef50951dc46"
format_stamp: "Formatted at 2026-06-20 20:08:40 on dist-test-slave-l8h4"
server_key: "810b863d7e37d6adba443cd848f74d7c"
server_key_iv: "05abd20c9fd890c4c455592d827e09ca"
server_key_version: "encryptionkey@0"
I20260620 20:08:40.576493 7566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:40.590534 7566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:40.593519 7566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:40.593694 7566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:40.595134 7566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:40.595183 7566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:40.595234 7566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:40.595258 7566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 20:08:40 dist-test-slave-l8h4 krb5kdc[7169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986120, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:40.605312 7566 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:43713
I20260620 20:08:40.605365 7696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:43713 every 8 connection(s)
I20260620 20:08:40.606362 7566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:08:40.608795 7263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.595937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:36735 (local address 127.5.91.190:40343)
0620 20:08:40.596063 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:40.596067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:40.597388 (+ 1321us) server_negotiation.cc:407] Connection header received
0620 20:08:40.598799 (+ 1411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:40.598802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:40.598881 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:40.598952 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:40.600232 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.600904 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.602077 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.602538 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.602633 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.605198 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:40.605201 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:40.605203 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:40.605227 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:40.606672 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:40.607234 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:40.607238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:40.607240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:40.607331 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:40.607727 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:40.607730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:40.607732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:40.607926 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:40.608012 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:40.608529 (+ 517us) server_negotiation.cc:299] Negotiation successful
0620 20:08:40.608633 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 20:08:40.609540 7590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.596446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:40343 (local address 127.5.91.131:36735)
0620 20:08:40.597222 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:40.597281 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:40.598560 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:40.599135 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:40.599144 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:40.599479 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:40.599992 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.600005 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.601042 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.601046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.601961 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.601972 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.602698 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.602702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.602868 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.603503 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:40.603524 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:40.605008 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:40.606779 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:40.606784 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:40.606799 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:40.607130 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:40.607465 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:40.607467 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:40.607469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:40.607614 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:40.608043 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:40.608048 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:40.608271 (+ 223us) client_negotiation.cc:769] Sending connection context
0620 20:08:40.608455 (+ 184us) client_negotiation.cc:240] Negotiation successful
0620 20:08:40.608705 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":132,"threads_started":1}
I20260620 20:08:40.610633 7697 heartbeater.cc:347] Connected to a master server at 127.5.91.190:40343
I20260620 20:08:40.610778 7697 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:40.611244 7697 heartbeater.cc:511] Master 127.5.91.190:40343 requested a full tablet report, sending...
I20260620 20:08:40.612284 7214 ts_manager.cc:194] Registered new tserver with Master: 2e5414be5a3c427595e36ef50951dc46 (127.5.91.131:43713)
I20260620 20:08:40.612958 7214 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:36735
I20260620 20:08:40.615746 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 7566
I20260620 20:08:40.615902 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:08:40.616725 5486 external_mini_cluster.cc:1428] Setting key ab21ac17541dfc87906e16f262dd6756
I20260620 20:08:40.619297 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 20:08:40.621794 7263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.612804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:50357 (local address 127.5.91.190:40343)
0620 20:08:40.612922 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:40.612926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:40.613077 (+ 151us) server_negotiation.cc:407] Connection header received
0620 20:08:40.613239 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:40.613242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:40.613316 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:40.613376 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:40.615103 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.615621 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.617272 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.617510 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.617598 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.618677 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:40.618680 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:40.618682 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:40.618703 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:40.620197 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:40.620614 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:40.620618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:40.620620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:40.620703 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:40.621020 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:40.621023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:40.621025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:40.621213 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:40.621300 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:40.621594 (+ 294us) server_negotiation.cc:299] Negotiation successful
0620 20:08:40.621683 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 20:08:40.621912 7703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.612707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:40343 (local address 127.5.91.131:50357)
0620 20:08:40.612995 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:40.613010 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:40.613135 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:40.614733 (+ 1598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:40.614735 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:40.614899 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:40.614990 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.614995 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.615851 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.615858 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.617161 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.617174 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.617617 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.617620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.617714 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.618221 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:40.618236 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:40.618549 (+ 313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:40.620315 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:40.620319 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:40.620321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:40.620526 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:40.620807 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:40.620810 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:40.620811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:40.620938 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:40.621325 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:40.621328 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:40.621434 (+ 106us) client_negotiation.cc:769] Sending connection context
0620 20:08:40.621545 (+ 111us) client_negotiation.cc:240] Negotiation successful
0620 20:08:40.621695 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":116,"threads_started":1}
Jun 20 20:08:40 dist-test-slave-l8h4 krb5kdc[7169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 20:08:40 dist-test-slave-l8h4 krb5kdc[7169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:40.647831 7263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.637418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51292 (local address 127.5.91.190:40343)
0620 20:08:40.637787 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:40.637791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:40.637806 (+ 15us) server_negotiation.cc:407] Connection header received
0620 20:08:40.637847 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:40.637850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:40.637890 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:40.637961 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:40.638753 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.639283 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.640149 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.640359 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.640458 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.642837 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:40.642840 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:40.642842 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:40.642869 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:40.645746 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:40.646494 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:40.646498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:40.646500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:40.646586 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:40.646909 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:40.646912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:40.646914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:40.647149 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:40.647278 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:40.647540 (+ 262us) server_negotiation.cc:299] Negotiation successful
0620 20:08:40.647663 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260620 20:08:40.651309 7214 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51292:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:08:40.655525 7214 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:08:40.670543 7717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.665073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:40759 (local address 127.0.0.1:45938)
0620 20:08:40.665949 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:40.666003 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:40.666159 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:40.666670 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:40.666676 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:40.666697 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:40.666836 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.666843 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.667807 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.667810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.668871 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.668881 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.669973 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.669977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.670093 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.670162 (+ 69us) client_negotiation.cc:769] Sending connection context
0620 20:08:40.670262 (+ 100us) client_negotiation.cc:240] Negotiation successful
0620 20:08:40.670373 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":824,"spinlock_wait_cycles":3584,"thread_start_us":85,"threads_started":1}
I20260620 20:08:40.670981 7718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.665821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45938 (local address 127.5.91.129:40759)
0620 20:08:40.666065 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:40.666073 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:40.666096 (+ 23us) server_negotiation.cc:407] Connection header received
0620 20:08:40.666312 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:40.666318 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:40.666528 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:40.666689 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:40.666923 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.667730 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.668979 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.669856 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.669954 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.670090 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:40.670735 (+ 645us) server_negotiation.cc:299] Negotiation successful
0620 20:08:40.670833 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":87,"threads_started":1}
I20260620 20:08:40.671495 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.665284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:43713 (local address 127.0.0.1:42288)
0620 20:08:40.666363 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:40.666404 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:40.666483 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:40.666888 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:40.666891 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:40.666911 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:40.667014 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.667020 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.668279 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.668283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.669464 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.669474 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.670559 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.670563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.670957 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.670985 (+ 28us) client_negotiation.cc:769] Sending connection context
0620 20:08:40.671285 (+ 300us) client_negotiation.cc:240] Negotiation successful
0620 20:08:40.671367 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1025,"thread_start_us":70,"threads_started":1}
I20260620 20:08:40.672390 7719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.666129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42288 (local address 127.5.91.131:43713)
0620 20:08:40.666444 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:40.666455 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:40.666476 (+ 21us) server_negotiation.cc:407] Connection header received
0620 20:08:40.666581 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:40.666586 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:40.666758 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:40.666922 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:40.667106 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.668158 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.669599 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.670418 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.671826 (+ 1408us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.672044 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:40.672109 (+ 65us) server_negotiation.cc:299] Negotiation successful
0620 20:08:40.672253 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":103,"threads_started":1}
I20260620 20:08:40.675057 7714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.665046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:41225 (local address 127.0.0.1:58318)
0620 20:08:40.665949 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:40.666003 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:40.666159 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:40.667482 (+ 1323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:40.667485 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:40.667502 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:40.667604 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.667609 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.670052 (+ 2443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.670055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.671112 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.671123 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.674676 (+ 3553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.674681 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.674785 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.674809 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 20:08:40.674861 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 20:08:40.674931 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"spinlock_wait_cycles":6144,"thread_start_us":73,"threads_started":1}
I20260620 20:08:40.674634 7351 tablet_service.cc:1467] Processing CreateTablet for tablet a6f8a4d3be3e4272accafc7c7ca1ee8f (DEFAULT_TABLE table=test-table [id=526705dbcc364982a6e1985eefbc44fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:40.675153 7631 tablet_service.cc:1467] Processing CreateTablet for tablet a6f8a4d3be3e4272accafc7c7ca1ee8f (DEFAULT_TABLE table=test-table [id=526705dbcc364982a6e1985eefbc44fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:40.675798 7351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6f8a4d3be3e4272accafc7c7ca1ee8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:40.676086 7716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.665180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58318 (local address 127.5.91.130:41225)
0620 20:08:40.667055 (+ 1875us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:40.667066 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:40.667086 (+ 20us) server_negotiation.cc:407] Connection header received
0620 20:08:40.667154 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:40.667158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:40.667322 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:40.667730 (+ 408us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:40.669081 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.669937 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.671429 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.674529 (+ 3100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.675480 (+ 951us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.675729 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:40.675783 (+ 54us) server_negotiation.cc:299] Negotiation successful
0620 20:08:40.675937 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1777,"thread_start_us":107,"threads_started":1}
I20260620 20:08:40.676117 7631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6f8a4d3be3e4272accafc7c7ca1ee8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:40.679009 7491 tablet_service.cc:1467] Processing CreateTablet for tablet a6f8a4d3be3e4272accafc7c7ca1ee8f (DEFAULT_TABLE table=test-table [id=526705dbcc364982a6e1985eefbc44fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:40.680295 7491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6f8a4d3be3e4272accafc7c7ca1ee8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:40.685360 7720 tablet_bootstrap.cc:492] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46: Bootstrap starting.
I20260620 20:08:40.688738 7720 tablet_bootstrap.cc:654] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:40.689642 7720 log.cc:824] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:40.691179 7722 tablet_bootstrap.cc:492] T a6f8a4d3be3e4272accafc7c7ca1ee8f P f390f74c2612442c903fc4ddca73be71: Bootstrap starting.
I20260620 20:08:40.692759 7723 tablet_bootstrap.cc:492] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 16b0d7dcc332467fb0bfad2531110705: Bootstrap starting.
I20260620 20:08:40.693220 7722 tablet_bootstrap.cc:654] T a6f8a4d3be3e4272accafc7c7ca1ee8f P f390f74c2612442c903fc4ddca73be71: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:40.693884 7722 log.cc:824] T a6f8a4d3be3e4272accafc7c7ca1ee8f P f390f74c2612442c903fc4ddca73be71: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:40.694644 7720 tablet_bootstrap.cc:492] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46: No bootstrap required, opened a new log
I20260620 20:08:40.694792 7723 tablet_bootstrap.cc:654] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 16b0d7dcc332467fb0bfad2531110705: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:40.694847 7720 ts_tablet_manager.cc:1397] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260620 20:08:40.695411 7722 tablet_bootstrap.cc:492] T a6f8a4d3be3e4272accafc7c7ca1ee8f P f390f74c2612442c903fc4ddca73be71: No bootstrap required, opened a new log
I20260620 20:08:40.695544 7723 log.cc:824] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 16b0d7dcc332467fb0bfad2531110705: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:40.695607 7722 ts_tablet_manager.cc:1397] T a6f8a4d3be3e4272accafc7c7ca1ee8f P f390f74c2612442c903fc4ddca73be71: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260620 20:08:40.697381 7723 tablet_bootstrap.cc:492] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 16b0d7dcc332467fb0bfad2531110705: No bootstrap required, opened a new log
I20260620 20:08:40.697559 7723 ts_tablet_manager.cc:1397] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 16b0d7dcc332467fb0bfad2531110705: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 20:08:40.698975 7720 raft_consensus.cc:348] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:40.699076 7722 raft_consensus.cc:348] T a6f8a4d3be3e4272accafc7c7ca1ee8f P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:40.699250 7720 raft_consensus.cc:374] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:40.699270 7722 raft_consensus.cc:374] T a6f8a4d3be3e4272accafc7c7ca1ee8f P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:40.699306 7720 raft_consensus.cc:729] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e5414be5a3c427595e36ef50951dc46, State: Initialized, Role: FOLLOWER
I20260620 20:08:40.699312 7722 raft_consensus.cc:729] T a6f8a4d3be3e4272accafc7c7ca1ee8f P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f390f74c2612442c903fc4ddca73be71, State: Initialized, Role: FOLLOWER
I20260620 20:08:40.699807 7720 consensus_queue.cc:260] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:40.699955 7722 consensus_queue.cc:260] T a6f8a4d3be3e4272accafc7c7ca1ee8f P f390f74c2612442c903fc4ddca73be71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:40.700626 7697 heartbeater.cc:503] Master 127.5.91.190:40343 was elected leader, sending a full tablet report...
I20260620 20:08:40.700850 7720 ts_tablet_manager.cc:1428] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260620 20:08:40.700948 7722 ts_tablet_manager.cc:1428] T a6f8a4d3be3e4272accafc7c7ca1ee8f P f390f74c2612442c903fc4ddca73be71: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260620 20:08:40.701260 7417 heartbeater.cc:503] Master 127.5.91.190:40343 was elected leader, sending a full tablet report...
I20260620 20:08:40.701339 7723 raft_consensus.cc:348] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:40.701563 7723 raft_consensus.cc:374] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:40.701617 7723 raft_consensus.cc:729] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16b0d7dcc332467fb0bfad2531110705, State: Initialized, Role: FOLLOWER
I20260620 20:08:40.702070 7723 consensus_queue.cc:260] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 16b0d7dcc332467fb0bfad2531110705 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:40.702704 7557 heartbeater.cc:503] Master 127.5.91.190:40343 was elected leader, sending a full tablet report...
I20260620 20:08:40.703023 7723 ts_tablet_manager.cc:1428] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 16b0d7dcc332467fb0bfad2531110705: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260620 20:08:40.763048 7726 raft_consensus.cc:482] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:08:40.763289 7726 raft_consensus.cc:504] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:40.764436 7726 leader_election.cc:288] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16b0d7dcc332467fb0bfad2531110705 (127.5.91.130:41225), f390f74c2612442c903fc4ddca73be71 (127.5.91.129:40759)
I20260620 20:08:40.768105 7716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.764759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:34033 (local address 127.5.91.130:41225)
0620 20:08:40.764882 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:40.764886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:40.765001 (+ 115us) server_negotiation.cc:407] Connection header received
0620 20:08:40.765147 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:40.765150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:40.765196 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:40.765293 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:40.765513 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.766022 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.767049 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.767576 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.767701 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.767755 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:40.767920 (+ 165us) server_negotiation.cc:299] Negotiation successful
0620 20:08:40.767986 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 20:08:40.768117 7590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.764702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:41225 (local address 127.5.91.131:34033)
0620 20:08:40.764914 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:40.764929 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:40.765037 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:40.765292 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:40.765295 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:40.765313 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:40.765421 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.765427 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.766119 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.766122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.766938 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.766946 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.767701 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.767705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.767808 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.767829 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 20:08:40.767897 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 20:08:40.767972 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138}
I20260620 20:08:40.768440 7718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.764788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:37443 (local address 127.5.91.129:40759)
0620 20:08:40.764998 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:40.765002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:40.765229 (+ 227us) server_negotiation.cc:407] Connection header received
0620 20:08:40.765317 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:40.765320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:40.765361 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:40.765423 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:40.766048 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.766571 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.767528 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.767922 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.768023 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.768070 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:40.768254 (+ 184us) server_negotiation.cc:299] Negotiation successful
0620 20:08:40.768328 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260620 20:08:40.768860 7729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.764714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:40759 (local address 127.5.91.131:37443)
0620 20:08:40.765148 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:40.765159 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:40.765237 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:40.765434 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:40.765436 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:40.765464 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:40.765941 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.765949 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.766667 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.766670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.767401 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.767410 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.768023 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.768026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.768119 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.768138 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 20:08:40.768662 (+ 524us) client_negotiation.cc:240] Negotiation successful
0620 20:08:40.768734 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":98,"thread_start_us":133,"threads_started":1}
I20260620 20:08:40.768846 7511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6f8a4d3be3e4272accafc7c7ca1ee8f" candidate_uuid: "2e5414be5a3c427595e36ef50951dc46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16b0d7dcc332467fb0bfad2531110705" is_pre_election: true
I20260620 20:08:40.769166 7511 raft_consensus.cc:2454] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e5414be5a3c427595e36ef50951dc46 in term 0.
I20260620 20:08:40.769435 7371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6f8a4d3be3e4272accafc7c7ca1ee8f" candidate_uuid: "2e5414be5a3c427595e36ef50951dc46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f390f74c2612442c903fc4ddca73be71" is_pre_election: true
I20260620 20:08:40.769677 7371 raft_consensus.cc:2454] T a6f8a4d3be3e4272accafc7c7ca1ee8f P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e5414be5a3c427595e36ef50951dc46 in term 0.
I20260620 20:08:40.769719 7585 leader_election.cc:302] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16b0d7dcc332467fb0bfad2531110705, 2e5414be5a3c427595e36ef50951dc46; no voters:
I20260620 20:08:40.770059 7726 raft_consensus.cc:2781] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:08:40.770162 7726 raft_consensus.cc:482] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:08:40.770232 7726 raft_consensus.cc:3037] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:40.771299 7726 raft_consensus.cc:504] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:40.771654 7726 leader_election.cc:288] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46 [CANDIDATE]: Term 1 election: Requested vote from peers 16b0d7dcc332467fb0bfad2531110705 (127.5.91.130:41225), f390f74c2612442c903fc4ddca73be71 (127.5.91.129:40759)
I20260620 20:08:40.771977 7371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6f8a4d3be3e4272accafc7c7ca1ee8f" candidate_uuid: "2e5414be5a3c427595e36ef50951dc46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f390f74c2612442c903fc4ddca73be71"
I20260620 20:08:40.772012 7511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6f8a4d3be3e4272accafc7c7ca1ee8f" candidate_uuid: "2e5414be5a3c427595e36ef50951dc46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16b0d7dcc332467fb0bfad2531110705"
I20260620 20:08:40.772092 7371 raft_consensus.cc:3037] T a6f8a4d3be3e4272accafc7c7ca1ee8f P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:40.772122 7511 raft_consensus.cc:3037] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:40.773232 7371 raft_consensus.cc:2454] T a6f8a4d3be3e4272accafc7c7ca1ee8f P f390f74c2612442c903fc4ddca73be71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e5414be5a3c427595e36ef50951dc46 in term 1.
I20260620 20:08:40.773538 7583 leader_election.cc:302] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e5414be5a3c427595e36ef50951dc46, f390f74c2612442c903fc4ddca73be71; no voters:
I20260620 20:08:40.773757 7726 raft_consensus.cc:2781] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:40.773757 7511 raft_consensus.cc:2454] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 16b0d7dcc332467fb0bfad2531110705 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e5414be5a3c427595e36ef50951dc46 in term 1.
I20260620 20:08:40.774026 7726 raft_consensus.cc:686] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46 [term 1 LEADER]: Becoming Leader. State: Replica: 2e5414be5a3c427595e36ef50951dc46, State: Running, Role: LEADER
I20260620 20:08:40.774430 7726 consensus_queue.cc:237] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:40.777618 7214 catalog_manager.cc:5495] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e5414be5a3c427595e36ef50951dc46 (127.5.91.131). New cstate: current_term: 1 leader_uuid: "2e5414be5a3c427595e36ef50951dc46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } health_report { overall_health: HEALTHY } } }
Jun 20 20:08:40 dist-test-slave-l8h4 krb5kdc[7169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986119, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:40.839486 7737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.827984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:40343 (local address 127.0.0.1:51308)
0620 20:08:40.828262 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:40.828276 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:40.828392 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:40.828607 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:40.828610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:40.829013 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:40.829101 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.829107 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.829800 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.829803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.830393 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.830401 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.830776 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.830779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.830909 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.834637 (+ 3728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:40.834650 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:40.836265 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:40.838044 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:40.838053 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:40.838058 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:40.838326 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:40.838563 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:40.838566 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:40.838567 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:40.838642 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:40.839000 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:40.839006 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:40.839107 (+ 101us) client_negotiation.cc:769] Sending connection context
0620 20:08:40.839184 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 20:08:40.839306 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":34,"thread_start_us":116,"threads_started":1}
I20260620 20:08:40.839496 7263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.828090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51308 (local address 127.5.91.190:40343)
0620 20:08:40.828212 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:40.828216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:40.828346 (+ 130us) server_negotiation.cc:407] Connection header received
0620 20:08:40.828475 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:40.828478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:40.828514 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:40.828574 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:40.829210 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.829708 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.830493 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.830687 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.830766 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.836435 (+ 5669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:40.836438 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:40.836440 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:40.836461 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:40.837935 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:40.838416 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:40.838419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:40.838420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:40.838490 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:40.838709 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:40.838711 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:40.838712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:40.838922 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:40.838995 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:40.839235 (+ 240us) server_negotiation.cc:299] Negotiation successful
0620 20:08:40.839358 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":43}
I20260620 20:08:40.843398 7214 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:51308:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 20:08:40.844141 7214 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:08:40.849339 7351 tablet_service.cc:1467] Processing CreateTablet for tablet 18d47ef1187e4bf4ade68e432af36528 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=419db933d7a8475d947da175b04c913b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 20:08:40.849354 7631 tablet_service.cc:1467] Processing CreateTablet for tablet 18d47ef1187e4bf4ade68e432af36528 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=419db933d7a8475d947da175b04c913b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 20:08:40.849354 7491 tablet_service.cc:1467] Processing CreateTablet for tablet 18d47ef1187e4bf4ade68e432af36528 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=419db933d7a8475d947da175b04c913b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 20:08:40.850033 7491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18d47ef1187e4bf4ade68e432af36528. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:40.850050 7631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18d47ef1187e4bf4ade68e432af36528. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:40.850049 7351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18d47ef1187e4bf4ade68e432af36528. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:40.852553 7722 tablet_bootstrap.cc:492] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71: Bootstrap starting.
I20260620 20:08:40.853170 7723 tablet_bootstrap.cc:492] T 18d47ef1187e4bf4ade68e432af36528 P 16b0d7dcc332467fb0bfad2531110705: Bootstrap starting.
I20260620 20:08:40.853482 7722 tablet_bootstrap.cc:654] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:40.853474 7720 tablet_bootstrap.cc:492] T 18d47ef1187e4bf4ade68e432af36528 P 2e5414be5a3c427595e36ef50951dc46: Bootstrap starting.
I20260620 20:08:40.854219 7723 tablet_bootstrap.cc:654] T 18d47ef1187e4bf4ade68e432af36528 P 16b0d7dcc332467fb0bfad2531110705: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:40.854396 7720 tablet_bootstrap.cc:654] T 18d47ef1187e4bf4ade68e432af36528 P 2e5414be5a3c427595e36ef50951dc46: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:40.854969 7722 tablet_bootstrap.cc:492] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71: No bootstrap required, opened a new log
I20260620 20:08:40.855075 7722 ts_tablet_manager.cc:1397] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260620 20:08:40.855449 7723 tablet_bootstrap.cc:492] T 18d47ef1187e4bf4ade68e432af36528 P 16b0d7dcc332467fb0bfad2531110705: No bootstrap required, opened a new log
I20260620 20:08:40.855628 7723 ts_tablet_manager.cc:1397] T 18d47ef1187e4bf4ade68e432af36528 P 16b0d7dcc332467fb0bfad2531110705: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260620 20:08:40.855734 7722 raft_consensus.cc:348] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:40.855870 7722 raft_consensus.cc:374] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:40.855916 7722 raft_consensus.cc:729] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f390f74c2612442c903fc4ddca73be71, State: Initialized, Role: FOLLOWER
I20260620 20:08:40.856052 7722 consensus_queue.cc:260] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:40.856127 7723 raft_consensus.cc:348] T 18d47ef1187e4bf4ade68e432af36528 P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:40.856213 7723 raft_consensus.cc:374] T 18d47ef1187e4bf4ade68e432af36528 P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:40.856240 7723 raft_consensus.cc:729] T 18d47ef1187e4bf4ade68e432af36528 P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16b0d7dcc332467fb0bfad2531110705, State: Initialized, Role: FOLLOWER
I20260620 20:08:40.856366 7722 ts_tablet_manager.cc:1428] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 20:08:40.856364 7723 consensus_queue.cc:260] T 18d47ef1187e4bf4ade68e432af36528 P 16b0d7dcc332467fb0bfad2531110705 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:40.856611 7723 ts_tablet_manager.cc:1428] T 18d47ef1187e4bf4ade68e432af36528 P 16b0d7dcc332467fb0bfad2531110705: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260620 20:08:40.856617 7727 tablet_replica.cc:439] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } } }
I20260620 20:08:40.856756 7727 tablet_replica.cc:442] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 20:08:40.857218 7728 tablet_replica.cc:439] T 18d47ef1187e4bf4ade68e432af36528 P 16b0d7dcc332467fb0bfad2531110705: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } } }
I20260620 20:08:40.857523 7728 tablet_replica.cc:442] T 18d47ef1187e4bf4ade68e432af36528 P 16b0d7dcc332467fb0bfad2531110705: This TxnStatusTablet replica's current role is: FOLLOWER
W20260620 20:08:40.857918 7698 tablet.cc:2369] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:08:40.858213 7720 tablet_bootstrap.cc:492] T 18d47ef1187e4bf4ade68e432af36528 P 2e5414be5a3c427595e36ef50951dc46: No bootstrap required, opened a new log
I20260620 20:08:40.858309 7720 ts_tablet_manager.cc:1397] T 18d47ef1187e4bf4ade68e432af36528 P 2e5414be5a3c427595e36ef50951dc46: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260620 20:08:40.858767 7720 raft_consensus.cc:348] T 18d47ef1187e4bf4ade68e432af36528 P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:40.858902 7720 raft_consensus.cc:374] T 18d47ef1187e4bf4ade68e432af36528 P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:40.858942 7720 raft_consensus.cc:729] T 18d47ef1187e4bf4ade68e432af36528 P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e5414be5a3c427595e36ef50951dc46, State: Initialized, Role: FOLLOWER
I20260620 20:08:40.859055 7720 consensus_queue.cc:260] T 18d47ef1187e4bf4ade68e432af36528 P 2e5414be5a3c427595e36ef50951dc46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:40.859292 7720 ts_tablet_manager.cc:1428] T 18d47ef1187e4bf4ade68e432af36528 P 2e5414be5a3c427595e36ef50951dc46: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 20:08:40.859267 7726 tablet_replica.cc:439] T 18d47ef1187e4bf4ade68e432af36528 P 2e5414be5a3c427595e36ef50951dc46: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } } }
I20260620 20:08:40.859385 7726 tablet_replica.cc:442] T 18d47ef1187e4bf4ade68e432af36528 P 2e5414be5a3c427595e36ef50951dc46: This TxnStatusTablet replica's current role is: FOLLOWER
W20260620 20:08:40.885191 7558 tablet.cc:2369] T 18d47ef1187e4bf4ade68e432af36528 P 16b0d7dcc332467fb0bfad2531110705: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 20:08:40.906863 7418 tablet.cc:2369] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:08:40.976876 7727 raft_consensus.cc:482] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:08:40.977025 7727 raft_consensus.cc:504] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:40.978065 7727 leader_election.cc:288] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16b0d7dcc332467fb0bfad2531110705 (127.5.91.130:41225), 2e5414be5a3c427595e36ef50951dc46 (127.5.91.131:43713)
I20260620 20:08:40.982112 7742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.978355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:41225 (local address 127.5.91.129:37121)
0620 20:08:40.978837 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:40.978850 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:40.978968 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:40.979197 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:40.979200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:40.979219 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:40.979341 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.979346 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.980109 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.980112 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.981032 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.981041 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.981665 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.981669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.981769 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.981791 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 20:08:40.981873 (+ 82us) client_negotiation.cc:240] Negotiation successful
0620 20:08:40.981945 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"spinlock_wait_cycles":1408,"thread_start_us":128,"threads_started":1}
I20260620 20:08:40.982111 7716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.978441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:37121 (local address 127.5.91.130:41225)
0620 20:08:40.978567 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:40.978571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:40.978935 (+ 364us) server_negotiation.cc:407] Connection header received
0620 20:08:40.979072 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:40.979075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:40.979116 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:40.979176 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:40.979438 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.980011 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.981145 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.981537 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.981641 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.981737 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:40.981906 (+ 169us) server_negotiation.cc:299] Negotiation successful
0620 20:08:40.981989 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 20:08:40.982858 7741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.978402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:43713 (local address 127.5.91.129:35849)
0620 20:08:40.978992 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:40.979011 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:40.979251 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:40.979549 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:40.979552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:40.979568 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:40.980021 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.980029 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.980747 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.980750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.981596 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:40.981606 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.982472 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.982475 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:40.982574 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.982593 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 20:08:40.982659 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 20:08:40.982720 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"spinlock_wait_cycles":13056,"thread_start_us":151,"threads_started":1}
I20260620 20:08:40.983155 7719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:40.978597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:35849 (local address 127.5.91.131:43713)
0620 20:08:40.978955 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:40.978961 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:40.979207 (+ 246us) server_negotiation.cc:407] Connection header received
0620 20:08:40.979413 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:40.979416 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:40.979466 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:40.979528 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:40.980120 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.980653 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.981725 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:40.982347 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:40.982435 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:40.982482 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:40.982821 (+ 339us) server_negotiation.cc:299] Negotiation successful
0620 20:08:40.982981 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194}
I20260620 20:08:40.983350 7511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18d47ef1187e4bf4ade68e432af36528" candidate_uuid: "f390f74c2612442c903fc4ddca73be71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16b0d7dcc332467fb0bfad2531110705" is_pre_election: true
I20260620 20:08:40.983489 7511 raft_consensus.cc:2454] T 18d47ef1187e4bf4ade68e432af36528 P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f390f74c2612442c903fc4ddca73be71 in term 0.
I20260620 20:08:40.983844 7305 leader_election.cc:302] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16b0d7dcc332467fb0bfad2531110705, f390f74c2612442c903fc4ddca73be71; no voters:
I20260620 20:08:40.983924 7651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18d47ef1187e4bf4ade68e432af36528" candidate_uuid: "f390f74c2612442c903fc4ddca73be71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e5414be5a3c427595e36ef50951dc46" is_pre_election: true
I20260620 20:08:40.984104 7727 raft_consensus.cc:2781] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:08:40.984179 7727 raft_consensus.cc:482] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:08:40.984223 7727 raft_consensus.cc:3037] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:40.984225 7651 raft_consensus.cc:2454] T 18d47ef1187e4bf4ade68e432af36528 P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f390f74c2612442c903fc4ddca73be71 in term 0.
I20260620 20:08:40.985288 7727 raft_consensus.cc:504] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:40.985620 7727 leader_election.cc:288] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71 [CANDIDATE]: Term 1 election: Requested vote from peers 16b0d7dcc332467fb0bfad2531110705 (127.5.91.130:41225), 2e5414be5a3c427595e36ef50951dc46 (127.5.91.131:43713)
I20260620 20:08:40.985986 7651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18d47ef1187e4bf4ade68e432af36528" candidate_uuid: "f390f74c2612442c903fc4ddca73be71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e5414be5a3c427595e36ef50951dc46"
I20260620 20:08:40.985986 7511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18d47ef1187e4bf4ade68e432af36528" candidate_uuid: "f390f74c2612442c903fc4ddca73be71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16b0d7dcc332467fb0bfad2531110705"
I20260620 20:08:40.986111 7651 raft_consensus.cc:3037] T 18d47ef1187e4bf4ade68e432af36528 P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:40.986111 7511 raft_consensus.cc:3037] T 18d47ef1187e4bf4ade68e432af36528 P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:40.987218 7511 raft_consensus.cc:2454] T 18d47ef1187e4bf4ade68e432af36528 P 16b0d7dcc332467fb0bfad2531110705 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f390f74c2612442c903fc4ddca73be71 in term 1.
I20260620 20:08:40.987236 7651 raft_consensus.cc:2454] T 18d47ef1187e4bf4ade68e432af36528 P 2e5414be5a3c427595e36ef50951dc46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f390f74c2612442c903fc4ddca73be71 in term 1.
I20260620 20:08:40.987530 7305 leader_election.cc:302] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16b0d7dcc332467fb0bfad2531110705, f390f74c2612442c903fc4ddca73be71; no voters:
I20260620 20:08:40.987722 7727 raft_consensus.cc:2781] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:40.987953 7727 raft_consensus.cc:686] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71 [term 1 LEADER]: Becoming Leader. State: Replica: f390f74c2612442c903fc4ddca73be71, State: Running, Role: LEADER
I20260620 20:08:40.988286 7727 consensus_queue.cc:237] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:40.990552 7743 tablet_replica.cc:439] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71: TxnStatusTablet state changed. Reason: New leader f390f74c2612442c903fc4ddca73be71. Latest consensus state: current_term: 1 leader_uuid: "f390f74c2612442c903fc4ddca73be71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } } }
I20260620 20:08:40.990669 7743 tablet_replica.cc:442] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71: This TxnStatusTablet replica's current role is: LEADER
I20260620 20:08:40.991148 7745 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 20:08:40.991364 7745 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 20:08:40.991650 7214 catalog_manager.cc:5495] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71 reported cstate change: term changed from 0 to 1, leader changed from <none> to f390f74c2612442c903fc4ddca73be71 (127.5.91.129). New cstate: current_term: 1 leader_uuid: "f390f74c2612442c903fc4ddca73be71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } health_report { overall_health: UNKNOWN } } }
I20260620 20:08:41.024690 7737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:41.020884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:40759 (local address 127.0.0.1:45954)
0620 20:08:41.021045 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:41.021061 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:41.021169 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:41.021478 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:41.021481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:41.021489 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 20:08:41.021590 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:41.021595 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:41.022461 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:41.022464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:41.023075 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:41.023083 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:41.023507 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:41.023511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:41.023600 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:41.023648 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 20:08:41.024379 (+ 731us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 20:08:41.024396 (+ 17us) client_negotiation.cc:769] Sending connection context
0620 20:08:41.024453 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 20:08:41.024548 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260620 20:08:41.024690 7718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:41.021027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45954 (local address 127.5.91.129:40759)
0620 20:08:41.021156 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:41.021161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:41.021175 (+ 14us) server_negotiation.cc:407] Connection header received
0620 20:08:41.021317 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:41.021320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:41.021371 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:41.021452 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:08:41.021680 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:41.022355 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:41.023178 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:41.023403 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:41.023490 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:41.023781 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:08:41.024278 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:08:41.024379 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:41.024473 (+ 94us) server_negotiation.cc:299] Negotiation successful
0620 20:08:41.024532 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 20:08:41.029726 7511 raft_consensus.cc:1261] T 18d47ef1187e4bf4ade68e432af36528 P 16b0d7dcc332467fb0bfad2531110705 [term 1 FOLLOWER]: Refusing update from remote peer f390f74c2612442c903fc4ddca73be71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:08:41.029726 7651 raft_consensus.cc:1261] T 18d47ef1187e4bf4ade68e432af36528 P 2e5414be5a3c427595e36ef50951dc46 [term 1 FOLLOWER]: Refusing update from remote peer f390f74c2612442c903fc4ddca73be71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:08:41.030457 7727 consensus_queue.cc:1035] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:08:41.030637 7743 consensus_queue.cc:1035] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:08:41.034109 7726 tablet_replica.cc:439] T 18d47ef1187e4bf4ade68e432af36528 P 2e5414be5a3c427595e36ef50951dc46: TxnStatusTablet state changed. Reason: New leader f390f74c2612442c903fc4ddca73be71. Latest consensus state: current_term: 1 leader_uuid: "f390f74c2612442c903fc4ddca73be71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } } }
I20260620 20:08:41.034564 7726 tablet_replica.cc:442] T 18d47ef1187e4bf4ade68e432af36528 P 2e5414be5a3c427595e36ef50951dc46: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 20:08:41.036298 7728 tablet_replica.cc:439] T 18d47ef1187e4bf4ade68e432af36528 P 16b0d7dcc332467fb0bfad2531110705: TxnStatusTablet state changed. Reason: New leader f390f74c2612442c903fc4ddca73be71. Latest consensus state: current_term: 1 leader_uuid: "f390f74c2612442c903fc4ddca73be71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } } }
I20260620 20:08:41.036445 7728 tablet_replica.cc:442] T 18d47ef1187e4bf4ade68e432af36528 P 16b0d7dcc332467fb0bfad2531110705: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 20:08:41.039225 7749 mvcc.cc:205] Tried to move back new op lower bound from 7299015151730876416 to 7299015151570784256. Current Snapshot: MvccSnapshot[applied={T|T < 7299015151730876416}]
I20260620 20:08:41.039388 7726 tablet_replica.cc:439] T 18d47ef1187e4bf4ade68e432af36528 P 2e5414be5a3c427595e36ef50951dc46: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f390f74c2612442c903fc4ddca73be71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } } }
I20260620 20:08:41.039563 7726 tablet_replica.cc:442] T 18d47ef1187e4bf4ade68e432af36528 P 2e5414be5a3c427595e36ef50951dc46: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 20:08:41.039635 7751 mvcc.cc:205] Tried to move back new op lower bound from 7299015151730876416 to 7299015151570784256. Current Snapshot: MvccSnapshot[applied={T|T < 7299015151730876416}]
I20260620 20:08:41.039805 7728 tablet_replica.cc:439] T 18d47ef1187e4bf4ade68e432af36528 P 16b0d7dcc332467fb0bfad2531110705: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f390f74c2612442c903fc4ddca73be71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } } }
I20260620 20:08:41.039896 7728 tablet_replica.cc:442] T 18d47ef1187e4bf4ade68e432af36528 P 16b0d7dcc332467fb0bfad2531110705: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 20:08:41.037901 7748 mvcc.cc:205] Tried to move back new op lower bound from 7299015151730876416 to 7299015151570784256. Current Snapshot: MvccSnapshot[applied={T|T < 7299015151730876416}]
I20260620 20:08:41.040688 7727 tablet_replica.cc:439] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f390f74c2612442c903fc4ddca73be71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } } }
I20260620 20:08:41.040794 7727 tablet_replica.cc:442] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71: This TxnStatusTablet replica's current role is: LEADER
I20260620 20:08:41.042217 7743 tablet_replica.cc:439] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f390f74c2612442c903fc4ddca73be71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } } }
I20260620 20:08:41.042325 7743 tablet_replica.cc:442] T 18d47ef1187e4bf4ade68e432af36528 P f390f74c2612442c903fc4ddca73be71: This TxnStatusTablet replica's current role is: LEADER
I20260620 20:08:41.050248 7719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:41.046329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42300 (local address 127.5.91.131:43713)
0620 20:08:41.046464 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:41.046467 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:41.046479 (+ 12us) server_negotiation.cc:407] Connection header received
0620 20:08:41.046602 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:41.046605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:41.046651 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:41.046734 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:08:41.046957 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:41.047467 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:41.048639 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:41.048880 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:41.049002 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:41.049255 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:08:41.049822 (+ 567us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:08:41.049948 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:41.050003 (+ 55us) server_negotiation.cc:299] Negotiation successful
0620 20:08:41.050063 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 20:08:41.061975 7703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:41.058807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:40759 (local address 127.5.91.131:33121)
0620 20:08:41.058956 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:41.058990 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:41.059082 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:41.059410 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:41.059414 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:41.059422 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 20:08:41.059533 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:41.059539 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:41.060220 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:41.060224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:41.060835 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:41.060843 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:41.061264 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:41.061267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:41.061349 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:41.061401 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 20:08:41.061689 (+ 288us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 20:08:41.061696 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 20:08:41.061754 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 20:08:41.061838 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260620 20:08:41.061975 7718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:41.059057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:33121 (local address 127.5.91.129:40759)
0620 20:08:41.059192 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:41.059196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:41.059211 (+ 15us) server_negotiation.cc:407] Connection header received
0620 20:08:41.059260 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:41.059262 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:41.059319 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:41.059382 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:08:41.059629 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:41.060124 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:41.060944 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:41.061167 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:41.061251 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:41.061543 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:08:41.061617 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:08:41.061673 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:41.061774 (+ 101us) server_negotiation.cc:299] Negotiation successful
0620 20:08:41.061826 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260620 20:08:41.070526 7511 raft_consensus.cc:1261] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 16b0d7dcc332467fb0bfad2531110705 [term 1 FOLLOWER]: Refusing update from remote peer 2e5414be5a3c427595e36ef50951dc46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:08:41.070791 7371 raft_consensus.cc:1261] T a6f8a4d3be3e4272accafc7c7ca1ee8f P f390f74c2612442c903fc4ddca73be71 [term 1 FOLLOWER]: Refusing update from remote peer 2e5414be5a3c427595e36ef50951dc46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:08:41.070958 7726 consensus_queue.cc:1035] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:08:41.071286 7730 consensus_queue.cc:1035] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:08:41.097105 7719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:41.093175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:34921 (local address 127.5.91.131:43713)
0620 20:08:41.093294 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:41.093299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:41.093502 (+ 203us) server_negotiation.cc:407] Connection header received
0620 20:08:41.093625 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:41.093628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:41.093679 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:41.093758 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:08:41.094160 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:41.094684 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:41.095965 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:41.096238 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:41.096344 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:41.096610 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:08:41.096701 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:08:41.096766 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:41.096912 (+ 146us) server_negotiation.cc:299] Negotiation successful
0620 20:08:41.096966 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 20:08:41.097529 7765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:41.093109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:43713 (local address 127.5.91.129:34921)
0620 20:08:41.093412 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:41.093428 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:41.093531 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:41.093892 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:41.093895 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:41.093904 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 20:08:41.094007 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:41.094013 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:41.094802 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:41.094805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:41.095811 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:41.095821 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:41.096345 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:41.096348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:41.096449 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:41.096494 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 20:08:41.096803 (+ 309us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 20:08:41.096809 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 20:08:41.097306 (+ 497us) client_negotiation.cc:240] Negotiation successful
0620 20:08:41.097396 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":104,"threads_started":1}
I20260620 20:08:41.131107 7213 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:08:41.131330 7213 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:08:41.134320 7213 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122: Sending DeleteTablet for 3 replicas of tablet a6f8a4d3be3e4272accafc7c7ca1ee8f
I20260620 20:08:41.135155 7491 tablet_service.cc:1514] Processing DeleteTablet for tablet a6f8a4d3be3e4272accafc7c7ca1ee8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:41 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:58318
I20260620 20:08:41.135764 7631 tablet_service.cc:1514] Processing DeleteTablet for tablet a6f8a4d3be3e4272accafc7c7ca1ee8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:41 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:42288
I20260620 20:08:41.136464 7213 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51292:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:08:41.136907 7213 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:08:41.136870 7351 tablet_service.cc:1514] Processing DeleteTablet for tablet a6f8a4d3be3e4272accafc7c7ca1ee8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:41 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:45938
I20260620 20:08:41.137435 7769 tablet_replica.cc:330] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46: stopping tablet replica
I20260620 20:08:41.137796 7769 raft_consensus.cc:2229] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46 [term 1 LEADER]: Raft consensus shutting down.
I20260620 20:08:41.138255 7769 raft_consensus.cc:2258] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:08:41.138896 7770 tablet_replica.cc:330] T a6f8a4d3be3e4272accafc7c7ca1ee8f P f390f74c2612442c903fc4ddca73be71: stopping tablet replica
I20260620 20:08:41.139173 7770 raft_consensus.cc:2229] T a6f8a4d3be3e4272accafc7c7ca1ee8f P f390f74c2612442c903fc4ddca73be71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:08:41.139351 7770 raft_consensus.cc:2258] T a6f8a4d3be3e4272accafc7c7ca1ee8f P f390f74c2612442c903fc4ddca73be71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:08:41.141767 7770 ts_tablet_manager.cc:1905] T a6f8a4d3be3e4272accafc7c7ca1ee8f P f390f74c2612442c903fc4ddca73be71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:08:41.142580 7351 tablet_service.cc:1467] Processing CreateTablet for tablet 5c621c09e7844c6ead8a701566a62075 (DEFAULT_TABLE table=test-table [id=73d9165961334177aa6180702471e10e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:41.142841 7351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c621c09e7844c6ead8a701566a62075. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:41.145100 7722 tablet_bootstrap.cc:492] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71: Bootstrap starting.
I20260620 20:08:41.145339 7631 tablet_service.cc:1467] Processing CreateTablet for tablet 5c621c09e7844c6ead8a701566a62075 (DEFAULT_TABLE table=test-table [id=73d9165961334177aa6180702471e10e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:41.145581 7631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c621c09e7844c6ead8a701566a62075. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:41.146080 7722 tablet_bootstrap.cc:654] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:41.147251 7491 tablet_service.cc:1467] Processing CreateTablet for tablet 5c621c09e7844c6ead8a701566a62075 (DEFAULT_TABLE table=test-table [id=73d9165961334177aa6180702471e10e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:41.146824 7770 ts_tablet_manager.cc:1918] T a6f8a4d3be3e4272accafc7c7ca1ee8f P f390f74c2612442c903fc4ddca73be71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 20:08:41.147394 7770 log.cc:1196] T a6f8a4d3be3e4272accafc7c7ca1ee8f P f390f74c2612442c903fc4ddca73be71: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-0/wal/wals/a6f8a4d3be3e4272accafc7c7ca1ee8f
I20260620 20:08:41.147500 7491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c621c09e7844c6ead8a701566a62075. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:41.147665 7770 ts_tablet_manager.cc:1939] T a6f8a4d3be3e4272accafc7c7ca1ee8f P f390f74c2612442c903fc4ddca73be71: Deleting consensus metadata
I20260620 20:08:41.148799 7199 catalog_manager.cc:4841] TS f390f74c2612442c903fc4ddca73be71 (127.5.91.129:40759): tablet a6f8a4d3be3e4272accafc7c7ca1ee8f (table test-table [id=526705dbcc364982a6e1985eefbc44fb]) successfully deleted
I20260620 20:08:41.149945 7723 tablet_bootstrap.cc:492] T 5c621c09e7844c6ead8a701566a62075 P 16b0d7dcc332467fb0bfad2531110705: Bootstrap starting.
I20260620 20:08:41.151242 7723 tablet_bootstrap.cc:654] T 5c621c09e7844c6ead8a701566a62075 P 16b0d7dcc332467fb0bfad2531110705: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:41.153621 7720 tablet_bootstrap.cc:492] T 5c621c09e7844c6ead8a701566a62075 P 2e5414be5a3c427595e36ef50951dc46: Bootstrap starting.
I20260620 20:08:41.154585 7720 tablet_bootstrap.cc:654] T 5c621c09e7844c6ead8a701566a62075 P 2e5414be5a3c427595e36ef50951dc46: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:41.155086 7769 ts_tablet_manager.cc:1905] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:08:41.155458 7722 tablet_bootstrap.cc:492] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71: No bootstrap required, opened a new log
I20260620 20:08:41.155536 7722 ts_tablet_manager.cc:1397] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260620 20:08:41.155997 7722 raft_consensus.cc:348] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:41.156112 7722 raft_consensus.cc:374] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:41.156140 7722 raft_consensus.cc:729] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f390f74c2612442c903fc4ddca73be71, State: Initialized, Role: FOLLOWER
I20260620 20:08:41.156267 7722 consensus_queue.cc:260] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:41.156498 7722 ts_tablet_manager.cc:1428] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 20:08:41.157220 7768 tablet_replica.cc:330] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 16b0d7dcc332467fb0bfad2531110705: stopping tablet replica
I20260620 20:08:41.157563 7768 raft_consensus.cc:2229] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 16b0d7dcc332467fb0bfad2531110705 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:08:41.157773 7768 raft_consensus.cc:2258] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 16b0d7dcc332467fb0bfad2531110705 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:08:41.159281 7768 ts_tablet_manager.cc:1905] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 16b0d7dcc332467fb0bfad2531110705: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:08:41.160506 7720 tablet_bootstrap.cc:492] T 5c621c09e7844c6ead8a701566a62075 P 2e5414be5a3c427595e36ef50951dc46: No bootstrap required, opened a new log
I20260620 20:08:41.160593 7720 ts_tablet_manager.cc:1397] T 5c621c09e7844c6ead8a701566a62075 P 2e5414be5a3c427595e36ef50951dc46: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260620 20:08:41.161032 7720 raft_consensus.cc:348] T 5c621c09e7844c6ead8a701566a62075 P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:41.161144 7720 raft_consensus.cc:374] T 5c621c09e7844c6ead8a701566a62075 P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:41.161171 7720 raft_consensus.cc:729] T 5c621c09e7844c6ead8a701566a62075 P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e5414be5a3c427595e36ef50951dc46, State: Initialized, Role: FOLLOWER
I20260620 20:08:41.161286 7720 consensus_queue.cc:260] T 5c621c09e7844c6ead8a701566a62075 P 2e5414be5a3c427595e36ef50951dc46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:41.161518 7720 ts_tablet_manager.cc:1428] T 5c621c09e7844c6ead8a701566a62075 P 2e5414be5a3c427595e36ef50951dc46: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 20:08:41.162168 7768 ts_tablet_manager.cc:1918] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 16b0d7dcc332467fb0bfad2531110705: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 20:08:41.162256 7768 log.cc:1196] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 16b0d7dcc332467fb0bfad2531110705: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-1/wal/wals/a6f8a4d3be3e4272accafc7c7ca1ee8f
I20260620 20:08:41.163053 7769 ts_tablet_manager.cc:1918] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 20:08:41.163144 7769 log.cc:1196] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/a6f8a4d3be3e4272accafc7c7ca1ee8f
I20260620 20:08:41.164067 7723 tablet_bootstrap.cc:492] T 5c621c09e7844c6ead8a701566a62075 P 16b0d7dcc332467fb0bfad2531110705: No bootstrap required, opened a new log
I20260620 20:08:41.164146 7723 ts_tablet_manager.cc:1397] T 5c621c09e7844c6ead8a701566a62075 P 16b0d7dcc332467fb0bfad2531110705: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.001s
I20260620 20:08:41.164548 7723 raft_consensus.cc:348] T 5c621c09e7844c6ead8a701566a62075 P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:41.164642 7723 raft_consensus.cc:374] T 5c621c09e7844c6ead8a701566a62075 P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:41.164672 7723 raft_consensus.cc:729] T 5c621c09e7844c6ead8a701566a62075 P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16b0d7dcc332467fb0bfad2531110705, State: Initialized, Role: FOLLOWER
I20260620 20:08:41.164793 7723 consensus_queue.cc:260] T 5c621c09e7844c6ead8a701566a62075 P 16b0d7dcc332467fb0bfad2531110705 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:41.165032 7723 ts_tablet_manager.cc:1428] T 5c621c09e7844c6ead8a701566a62075 P 16b0d7dcc332467fb0bfad2531110705: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 20:08:41.165601 7769 ts_tablet_manager.cc:1939] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 2e5414be5a3c427595e36ef50951dc46: Deleting consensus metadata
I20260620 20:08:41.166882 7200 catalog_manager.cc:4841] TS 2e5414be5a3c427595e36ef50951dc46 (127.5.91.131:43713): tablet a6f8a4d3be3e4272accafc7c7ca1ee8f (table test-table [id=526705dbcc364982a6e1985eefbc44fb]) successfully deleted
I20260620 20:08:41.167037 7768 ts_tablet_manager.cc:1939] T a6f8a4d3be3e4272accafc7c7ca1ee8f P 16b0d7dcc332467fb0bfad2531110705: Deleting consensus metadata
I20260620 20:08:41.168150 7201 catalog_manager.cc:4841] TS 16b0d7dcc332467fb0bfad2531110705 (127.5.91.130:41225): tablet a6f8a4d3be3e4272accafc7c7ca1ee8f (table test-table [id=526705dbcc364982a6e1985eefbc44fb]) successfully deleted
I20260620 20:08:41.170833 7743 raft_consensus.cc:482] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:08:41.170943 7743 raft_consensus.cc:504] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:41.171399 7743 leader_election.cc:288] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16b0d7dcc332467fb0bfad2531110705 (127.5.91.130:41225), 2e5414be5a3c427595e36ef50951dc46 (127.5.91.131:43713)
I20260620 20:08:41.171705 7511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c621c09e7844c6ead8a701566a62075" candidate_uuid: "f390f74c2612442c903fc4ddca73be71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16b0d7dcc332467fb0bfad2531110705" is_pre_election: true
I20260620 20:08:41.171849 7511 raft_consensus.cc:2454] T 5c621c09e7844c6ead8a701566a62075 P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f390f74c2612442c903fc4ddca73be71 in term 0.
I20260620 20:08:41.172039 7651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c621c09e7844c6ead8a701566a62075" candidate_uuid: "f390f74c2612442c903fc4ddca73be71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e5414be5a3c427595e36ef50951dc46" is_pre_election: true
I20260620 20:08:41.172143 7651 raft_consensus.cc:2454] T 5c621c09e7844c6ead8a701566a62075 P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f390f74c2612442c903fc4ddca73be71 in term 0.
I20260620 20:08:41.172508 7305 leader_election.cc:302] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16b0d7dcc332467fb0bfad2531110705, f390f74c2612442c903fc4ddca73be71; no voters:
I20260620 20:08:41.172804 7743 raft_consensus.cc:2781] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:08:41.172864 7743 raft_consensus.cc:482] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:08:41.172897 7743 raft_consensus.cc:3037] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:41.173995 7743 raft_consensus.cc:504] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:41.174341 7743 leader_election.cc:288] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71 [CANDIDATE]: Term 1 election: Requested vote from peers 16b0d7dcc332467fb0bfad2531110705 (127.5.91.130:41225), 2e5414be5a3c427595e36ef50951dc46 (127.5.91.131:43713)
I20260620 20:08:41.174768 7511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c621c09e7844c6ead8a701566a62075" candidate_uuid: "f390f74c2612442c903fc4ddca73be71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16b0d7dcc332467fb0bfad2531110705"
I20260620 20:08:41.174944 7511 raft_consensus.cc:3037] T 5c621c09e7844c6ead8a701566a62075 P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:41.175849 7511 raft_consensus.cc:2454] T 5c621c09e7844c6ead8a701566a62075 P 16b0d7dcc332467fb0bfad2531110705 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f390f74c2612442c903fc4ddca73be71 in term 1.
I20260620 20:08:41.176148 7305 leader_election.cc:302] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16b0d7dcc332467fb0bfad2531110705, f390f74c2612442c903fc4ddca73be71; no voters:
I20260620 20:08:41.176332 7743 raft_consensus.cc:2781] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:41.176409 7743 raft_consensus.cc:686] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71 [term 1 LEADER]: Becoming Leader. State: Replica: f390f74c2612442c903fc4ddca73be71, State: Running, Role: LEADER
I20260620 20:08:41.176535 7743 consensus_queue.cc:237] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:41.177275 7651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c621c09e7844c6ead8a701566a62075" candidate_uuid: "f390f74c2612442c903fc4ddca73be71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e5414be5a3c427595e36ef50951dc46"
I20260620 20:08:41.177450 7651 raft_consensus.cc:3037] T 5c621c09e7844c6ead8a701566a62075 P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:41.178311 7214 catalog_manager.cc:5495] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71 reported cstate change: term changed from 0 to 1, leader changed from <none> to f390f74c2612442c903fc4ddca73be71 (127.5.91.129). New cstate: current_term: 1 leader_uuid: "f390f74c2612442c903fc4ddca73be71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } health_report { overall_health: UNKNOWN } } }
I20260620 20:08:41.178571 7651 raft_consensus.cc:2454] T 5c621c09e7844c6ead8a701566a62075 P 2e5414be5a3c427595e36ef50951dc46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f390f74c2612442c903fc4ddca73be71 in term 1.
I20260620 20:08:41.200398 7718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:41.196923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45964 (local address 127.5.91.129:40759)
0620 20:08:41.197026 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:41.197030 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:41.197045 (+ 15us) server_negotiation.cc:407] Connection header received
0620 20:08:41.197085 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:41.197088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:41.197135 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:41.197221 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:08:41.197597 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:41.198262 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:41.199400 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:41.199610 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:41.199695 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:41.199992 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:08:41.200065 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:08:41.200129 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:41.200213 (+ 84us) server_negotiation.cc:299] Negotiation successful
0620 20:08:41.200277 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 20:08:41.210256 7718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:41.206127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:57021 (local address 127.5.91.129:40759)
0620 20:08:41.206257 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:41.206261 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:41.206274 (+ 13us) server_negotiation.cc:407] Connection header received
0620 20:08:41.206369 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:41.206372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:41.206418 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:41.206491 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:08:41.206684 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:41.207559 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:41.208792 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:41.209128 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:41.209241 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:41.209495 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:08:41.209579 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:08:41.209646 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:41.210009 (+ 363us) server_negotiation.cc:299] Negotiation successful
0620 20:08:41.210065 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":42}
I20260620 20:08:41.210430 7765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:41.206069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:40759 (local address 127.5.91.129:57021)
0620 20:08:41.206198 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:41.206213 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:41.206279 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:41.206510 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:41.206512 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:41.206518 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 20:08:41.206604 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:41.206608 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:41.207903 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:41.207907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:41.208666 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:41.208676 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:41.209287 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:41.209290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:41.209364 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:41.209390 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 20:08:41.209839 (+ 449us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 20:08:41.209853 (+ 14us) client_negotiation.cc:769] Sending connection context
0620 20:08:41.209987 (+ 134us) client_negotiation.cc:240] Negotiation successful
0620 20:08:41.210185 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260620 20:08:41.216639 7511 raft_consensus.cc:1261] T 5c621c09e7844c6ead8a701566a62075 P 16b0d7dcc332467fb0bfad2531110705 [term 1 FOLLOWER]: Refusing update from remote peer f390f74c2612442c903fc4ddca73be71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:08:41.216629 7651 raft_consensus.cc:1261] T 5c621c09e7844c6ead8a701566a62075 P 2e5414be5a3c427595e36ef50951dc46 [term 1 FOLLOWER]: Refusing update from remote peer f390f74c2612442c903fc4ddca73be71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:08:41.217015 7757 consensus_queue.cc:1035] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:08:41.217175 7727 consensus_queue.cc:1035] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 20:08:41.235918 7748 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260620 20:08:42.802284 7213 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:08:42.802433 7213 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:08:42.805521 7213 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122: Sending DeleteTablet for 3 replicas of tablet 5c621c09e7844c6ead8a701566a62075
I20260620 20:08:42.806094 7351 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c621c09e7844c6ead8a701566a62075 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:42 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:45938
I20260620 20:08:42.806167 7631 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c621c09e7844c6ead8a701566a62075 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:42 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:42288
I20260620 20:08:42.806429 7491 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c621c09e7844c6ead8a701566a62075 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:42 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:58318
I20260620 20:08:42.806525 7799 tablet_replica.cc:330] T 5c621c09e7844c6ead8a701566a62075 P 2e5414be5a3c427595e36ef50951dc46: stopping tablet replica
I20260620 20:08:42.806656 7799 raft_consensus.cc:2229] T 5c621c09e7844c6ead8a701566a62075 P 2e5414be5a3c427595e36ef50951dc46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:08:42.806763 7798 tablet_replica.cc:330] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71: stopping tablet replica
I20260620 20:08:42.806779 7799 raft_consensus.cc:2258] T 5c621c09e7844c6ead8a701566a62075 P 2e5414be5a3c427595e36ef50951dc46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:08:42.806797 7800 tablet_replica.cc:330] T 5c621c09e7844c6ead8a701566a62075 P 16b0d7dcc332467fb0bfad2531110705: stopping tablet replica
I20260620 20:08:42.806885 7798 raft_consensus.cc:2229] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71 [term 1 LEADER]: Raft consensus shutting down.
I20260620 20:08:42.807029 7800 raft_consensus.cc:2229] T 5c621c09e7844c6ead8a701566a62075 P 16b0d7dcc332467fb0bfad2531110705 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:08:42.807179 7800 raft_consensus.cc:2258] T 5c621c09e7844c6ead8a701566a62075 P 16b0d7dcc332467fb0bfad2531110705 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:08:42.807192 7798 raft_consensus.cc:2258] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:08:42.807433 7799 ts_tablet_manager.cc:1905] T 5c621c09e7844c6ead8a701566a62075 P 2e5414be5a3c427595e36ef50951dc46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:08:42.807749 7798 ts_tablet_manager.cc:1905] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:08:42.810293 7799 ts_tablet_manager.cc:1918] T 5c621c09e7844c6ead8a701566a62075 P 2e5414be5a3c427595e36ef50951dc46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 20:08:42.810376 7799 log.cc:1196] T 5c621c09e7844c6ead8a701566a62075 P 2e5414be5a3c427595e36ef50951dc46: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/5c621c09e7844c6ead8a701566a62075
I20260620 20:08:42.810592 7799 ts_tablet_manager.cc:1939] T 5c621c09e7844c6ead8a701566a62075 P 2e5414be5a3c427595e36ef50951dc46: Deleting consensus metadata
I20260620 20:08:42.811302 7200 catalog_manager.cc:4841] TS 2e5414be5a3c427595e36ef50951dc46 (127.5.91.131:43713): tablet 5c621c09e7844c6ead8a701566a62075 (table test-table [id=73d9165961334177aa6180702471e10e]) successfully deleted
I20260620 20:08:42.812094 7800 ts_tablet_manager.cc:1905] T 5c621c09e7844c6ead8a701566a62075 P 16b0d7dcc332467fb0bfad2531110705: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:08:42.812696 7798 ts_tablet_manager.cc:1918] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 20:08:42.812824 7798 log.cc:1196] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-0/wal/wals/5c621c09e7844c6ead8a701566a62075
I20260620 20:08:42.813159 7798 ts_tablet_manager.cc:1939] T 5c621c09e7844c6ead8a701566a62075 P f390f74c2612442c903fc4ddca73be71: Deleting consensus metadata
I20260620 20:08:42.814199 7800 ts_tablet_manager.cc:1918] T 5c621c09e7844c6ead8a701566a62075 P 16b0d7dcc332467fb0bfad2531110705: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 20:08:42.814278 7800 log.cc:1196] T 5c621c09e7844c6ead8a701566a62075 P 16b0d7dcc332467fb0bfad2531110705: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-1/wal/wals/5c621c09e7844c6ead8a701566a62075
I20260620 20:08:42.814283 7199 catalog_manager.cc:4841] TS f390f74c2612442c903fc4ddca73be71 (127.5.91.129:40759): tablet 5c621c09e7844c6ead8a701566a62075 (table test-table [id=73d9165961334177aa6180702471e10e]) successfully deleted
I20260620 20:08:42.814569 7800 ts_tablet_manager.cc:1939] T 5c621c09e7844c6ead8a701566a62075 P 16b0d7dcc332467fb0bfad2531110705: Deleting consensus metadata
I20260620 20:08:42.815414 7201 catalog_manager.cc:4841] TS 16b0d7dcc332467fb0bfad2531110705 (127.5.91.130:41225): tablet 5c621c09e7844c6ead8a701566a62075 (table test-table [id=73d9165961334177aa6180702471e10e]) successfully deleted
Jun 20 20:08:42 dist-test-slave-l8h4 krb5kdc[7169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986122, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 20:08:42 dist-test-slave-l8h4 krb5kdc[7169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986122, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:42.841276 7808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:42.831282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47198 (local address 127.5.91.190:40343)
0620 20:08:42.831560 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:42.831569 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:42.831586 (+ 17us) server_negotiation.cc:407] Connection header received
0620 20:08:42.831631 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:42.831634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:42.831681 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:42.831773 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:42.832833 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:42.833538 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:42.834591 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:42.834839 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:42.834934 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:42.837841 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:42.837845 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:42.837847 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:42.837874 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:42.839296 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:42.839836 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:42.839839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:42.839840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:42.840032 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:42.840365 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:42.840368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:42.840369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:42.840584 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:42.840695 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:42.840968 (+ 273us) server_negotiation.cc:299] Negotiation successful
0620 20:08:42.841076 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":99,"threads_started":1}
I20260620 20:08:42.844240 7213 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47198:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:08:42.844787 7213 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:08:42.849561 7351 tablet_service.cc:1467] Processing CreateTablet for tablet c360b3942d00499880b2bfa0b7bb4ea8 (DEFAULT_TABLE table=test-table [id=3be58799487b4d4a8f03eca4fd5558a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:42.849596 7491 tablet_service.cc:1467] Processing CreateTablet for tablet c360b3942d00499880b2bfa0b7bb4ea8 (DEFAULT_TABLE table=test-table [id=3be58799487b4d4a8f03eca4fd5558a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:42.849658 7631 tablet_service.cc:1467] Processing CreateTablet for tablet c360b3942d00499880b2bfa0b7bb4ea8 (DEFAULT_TABLE table=test-table [id=3be58799487b4d4a8f03eca4fd5558a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:42.849881 7491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c360b3942d00499880b2bfa0b7bb4ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:42.849881 7351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c360b3942d00499880b2bfa0b7bb4ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:42.849936 7631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c360b3942d00499880b2bfa0b7bb4ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:42.852176 7809 tablet_bootstrap.cc:492] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46: Bootstrap starting.
I20260620 20:08:42.853165 7809 tablet_bootstrap.cc:654] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:42.853609 7810 tablet_bootstrap.cc:492] T c360b3942d00499880b2bfa0b7bb4ea8 P 16b0d7dcc332467fb0bfad2531110705: Bootstrap starting.
I20260620 20:08:42.853741 7811 tablet_bootstrap.cc:492] T c360b3942d00499880b2bfa0b7bb4ea8 P f390f74c2612442c903fc4ddca73be71: Bootstrap starting.
I20260620 20:08:42.854578 7809 tablet_bootstrap.cc:492] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46: No bootstrap required, opened a new log
I20260620 20:08:42.854640 7811 tablet_bootstrap.cc:654] T c360b3942d00499880b2bfa0b7bb4ea8 P f390f74c2612442c903fc4ddca73be71: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:42.854677 7809 ts_tablet_manager.cc:1397] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 20:08:42.854609 7810 tablet_bootstrap.cc:654] T c360b3942d00499880b2bfa0b7bb4ea8 P 16b0d7dcc332467fb0bfad2531110705: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:42.855159 7809 raft_consensus.cc:348] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:42.855285 7809 raft_consensus.cc:374] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:42.855325 7809 raft_consensus.cc:729] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e5414be5a3c427595e36ef50951dc46, State: Initialized, Role: FOLLOWER
I20260620 20:08:42.855440 7809 consensus_queue.cc:260] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:42.855795 7809 ts_tablet_manager.cc:1428] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 20:08:42.856336 7811 tablet_bootstrap.cc:492] T c360b3942d00499880b2bfa0b7bb4ea8 P f390f74c2612442c903fc4ddca73be71: No bootstrap required, opened a new log
I20260620 20:08:42.856419 7811 ts_tablet_manager.cc:1397] T c360b3942d00499880b2bfa0b7bb4ea8 P f390f74c2612442c903fc4ddca73be71: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 20:08:42.856850 7810 tablet_bootstrap.cc:492] T c360b3942d00499880b2bfa0b7bb4ea8 P 16b0d7dcc332467fb0bfad2531110705: No bootstrap required, opened a new log
I20260620 20:08:42.856828 7811 raft_consensus.cc:348] T c360b3942d00499880b2bfa0b7bb4ea8 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:42.856932 7810 ts_tablet_manager.cc:1397] T c360b3942d00499880b2bfa0b7bb4ea8 P 16b0d7dcc332467fb0bfad2531110705: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 20:08:42.856940 7811 raft_consensus.cc:374] T c360b3942d00499880b2bfa0b7bb4ea8 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:42.856969 7811 raft_consensus.cc:729] T c360b3942d00499880b2bfa0b7bb4ea8 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f390f74c2612442c903fc4ddca73be71, State: Initialized, Role: FOLLOWER
I20260620 20:08:42.857087 7811 consensus_queue.cc:260] T c360b3942d00499880b2bfa0b7bb4ea8 P f390f74c2612442c903fc4ddca73be71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:42.857343 7811 ts_tablet_manager.cc:1428] T c360b3942d00499880b2bfa0b7bb4ea8 P f390f74c2612442c903fc4ddca73be71: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 20:08:42.857386 7810 raft_consensus.cc:348] T c360b3942d00499880b2bfa0b7bb4ea8 P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:42.857542 7810 raft_consensus.cc:374] T c360b3942d00499880b2bfa0b7bb4ea8 P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:42.857582 7810 raft_consensus.cc:729] T c360b3942d00499880b2bfa0b7bb4ea8 P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16b0d7dcc332467fb0bfad2531110705, State: Initialized, Role: FOLLOWER
I20260620 20:08:42.857717 7810 consensus_queue.cc:260] T c360b3942d00499880b2bfa0b7bb4ea8 P 16b0d7dcc332467fb0bfad2531110705 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:42.858086 7810 ts_tablet_manager.cc:1428] T c360b3942d00499880b2bfa0b7bb4ea8 P 16b0d7dcc332467fb0bfad2531110705: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 20:08:43.034541 7814 raft_consensus.cc:482] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:08:43.034735 7814 raft_consensus.cc:504] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:43.035209 7814 leader_election.cc:288] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f390f74c2612442c903fc4ddca73be71 (127.5.91.129:40759), 16b0d7dcc332467fb0bfad2531110705 (127.5.91.130:41225)
I20260620 20:08:43.035655 7371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c360b3942d00499880b2bfa0b7bb4ea8" candidate_uuid: "2e5414be5a3c427595e36ef50951dc46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f390f74c2612442c903fc4ddca73be71" is_pre_election: true
I20260620 20:08:43.035751 7510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c360b3942d00499880b2bfa0b7bb4ea8" candidate_uuid: "2e5414be5a3c427595e36ef50951dc46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16b0d7dcc332467fb0bfad2531110705" is_pre_election: true
I20260620 20:08:43.035794 7371 raft_consensus.cc:2454] T c360b3942d00499880b2bfa0b7bb4ea8 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e5414be5a3c427595e36ef50951dc46 in term 0.
I20260620 20:08:43.035890 7510 raft_consensus.cc:2454] T c360b3942d00499880b2bfa0b7bb4ea8 P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e5414be5a3c427595e36ef50951dc46 in term 0.
I20260620 20:08:43.036118 7583 leader_election.cc:302] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e5414be5a3c427595e36ef50951dc46, f390f74c2612442c903fc4ddca73be71; no voters:
I20260620 20:08:43.036291 7814 raft_consensus.cc:2781] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:08:43.036351 7814 raft_consensus.cc:482] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:08:43.036391 7814 raft_consensus.cc:3037] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:43.037370 7814 raft_consensus.cc:504] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:43.037698 7814 leader_election.cc:288] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46 [CANDIDATE]: Term 1 election: Requested vote from peers f390f74c2612442c903fc4ddca73be71 (127.5.91.129:40759), 16b0d7dcc332467fb0bfad2531110705 (127.5.91.130:41225)
I20260620 20:08:43.038285 7510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c360b3942d00499880b2bfa0b7bb4ea8" candidate_uuid: "2e5414be5a3c427595e36ef50951dc46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16b0d7dcc332467fb0bfad2531110705"
I20260620 20:08:43.038247 7371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c360b3942d00499880b2bfa0b7bb4ea8" candidate_uuid: "2e5414be5a3c427595e36ef50951dc46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f390f74c2612442c903fc4ddca73be71"
I20260620 20:08:43.038461 7371 raft_consensus.cc:3037] T c360b3942d00499880b2bfa0b7bb4ea8 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:43.038461 7510 raft_consensus.cc:3037] T c360b3942d00499880b2bfa0b7bb4ea8 P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:43.039865 7510 raft_consensus.cc:2454] T c360b3942d00499880b2bfa0b7bb4ea8 P 16b0d7dcc332467fb0bfad2531110705 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e5414be5a3c427595e36ef50951dc46 in term 1.
I20260620 20:08:43.039865 7371 raft_consensus.cc:2454] T c360b3942d00499880b2bfa0b7bb4ea8 P f390f74c2612442c903fc4ddca73be71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e5414be5a3c427595e36ef50951dc46 in term 1.
I20260620 20:08:43.040169 7585 leader_election.cc:302] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16b0d7dcc332467fb0bfad2531110705, 2e5414be5a3c427595e36ef50951dc46; no voters:
I20260620 20:08:43.040364 7814 raft_consensus.cc:2781] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:43.040586 7814 raft_consensus.cc:686] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46 [term 1 LEADER]: Becoming Leader. State: Replica: 2e5414be5a3c427595e36ef50951dc46, State: Running, Role: LEADER
I20260620 20:08:43.040742 7814 consensus_queue.cc:237] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:43.042959 7214 catalog_manager.cc:5495] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e5414be5a3c427595e36ef50951dc46 (127.5.91.131). New cstate: current_term: 1 leader_uuid: "2e5414be5a3c427595e36ef50951dc46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } health_report { overall_health: HEALTHY } } }
I20260620 20:08:43.105396 7820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:43.101754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49466 (local address 127.5.91.131:43713)
0620 20:08:43.102062 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:43.102068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:43.102081 (+ 13us) server_negotiation.cc:407] Connection header received
0620 20:08:43.102132 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:43.102162 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:43.102211 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:43.102290 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:08:43.102586 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:43.103164 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:43.104026 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:43.104274 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:43.104364 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:43.104765 (+ 401us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:08:43.104838 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:08:43.104934 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:43.105196 (+ 262us) server_negotiation.cc:299] Negotiation successful
0620 20:08:43.105247 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":144,"threads_started":1}
I20260620 20:08:43.111999 7371 raft_consensus.cc:1261] T c360b3942d00499880b2bfa0b7bb4ea8 P f390f74c2612442c903fc4ddca73be71 [term 1 FOLLOWER]: Refusing update from remote peer 2e5414be5a3c427595e36ef50951dc46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:08:43.111999 7510 raft_consensus.cc:1261] T c360b3942d00499880b2bfa0b7bb4ea8 P 16b0d7dcc332467fb0bfad2531110705 [term 1 FOLLOWER]: Refusing update from remote peer 2e5414be5a3c427595e36ef50951dc46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:08:43.112433 7817 consensus_queue.cc:1035] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:08:43.112684 7814 consensus_queue.cc:1035] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:08:43.151294 7214 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:08:43.151464 7214 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:08:43.153909 7214 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122: Sending DeleteTablet for 3 replicas of tablet c360b3942d00499880b2bfa0b7bb4ea8
I20260620 20:08:43.154861 7351 tablet_service.cc:1514] Processing DeleteTablet for tablet c360b3942d00499880b2bfa0b7bb4ea8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:43 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:45938
I20260620 20:08:43.154856 7491 tablet_service.cc:1514] Processing DeleteTablet for tablet c360b3942d00499880b2bfa0b7bb4ea8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:43 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:58318
I20260620 20:08:43.155107 7798 tablet_replica.cc:330] T c360b3942d00499880b2bfa0b7bb4ea8 P f390f74c2612442c903fc4ddca73be71: stopping tablet replica
I20260620 20:08:43.155102 7800 tablet_replica.cc:330] T c360b3942d00499880b2bfa0b7bb4ea8 P 16b0d7dcc332467fb0bfad2531110705: stopping tablet replica
I20260620 20:08:43.155221 7798 raft_consensus.cc:2229] T c360b3942d00499880b2bfa0b7bb4ea8 P f390f74c2612442c903fc4ddca73be71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:08:43.155258 7800 raft_consensus.cc:2229] T c360b3942d00499880b2bfa0b7bb4ea8 P 16b0d7dcc332467fb0bfad2531110705 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:08:43.155326 7798 raft_consensus.cc:2258] T c360b3942d00499880b2bfa0b7bb4ea8 P f390f74c2612442c903fc4ddca73be71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:08:43.155390 7800 raft_consensus.cc:2258] T c360b3942d00499880b2bfa0b7bb4ea8 P 16b0d7dcc332467fb0bfad2531110705 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:08:43.155788 7211 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47198:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 20:08:43.155860 7631 tablet_service.cc:1514] Processing DeleteTablet for tablet c360b3942d00499880b2bfa0b7bb4ea8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:43 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:42288
I20260620 20:08:43.156028 7799 tablet_replica.cc:330] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46: stopping tablet replica
I20260620 20:08:43.156131 7799 raft_consensus.cc:2229] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46 [term 1 LEADER]: Raft consensus shutting down.
W20260620 20:08:43.156227 7211 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:08:43.156332 7799 raft_consensus.cc:2258] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:08:43.156368 7800 ts_tablet_manager.cc:1905] T c360b3942d00499880b2bfa0b7bb4ea8 P 16b0d7dcc332467fb0bfad2531110705: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:08:43.157087 7798 ts_tablet_manager.cc:1905] T c360b3942d00499880b2bfa0b7bb4ea8 P f390f74c2612442c903fc4ddca73be71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:08:43.159376 7799 ts_tablet_manager.cc:1905] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:08:43.164057 7351 tablet_service.cc:1467] Processing CreateTablet for tablet 687984ab33f14923b56b4207159078e9 (DEFAULT_TABLE table=test-table [id=3bac536ab248439e9e86118ee2562a94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:43.164206 7491 tablet_service.cc:1467] Processing CreateTablet for tablet 687984ab33f14923b56b4207159078e9 (DEFAULT_TABLE table=test-table [id=3bac536ab248439e9e86118ee2562a94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:43.164285 7631 tablet_service.cc:1467] Processing CreateTablet for tablet 687984ab33f14923b56b4207159078e9 (DEFAULT_TABLE table=test-table [id=3bac536ab248439e9e86118ee2562a94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:43.171530 7491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 687984ab33f14923b56b4207159078e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:43.171561 7800 ts_tablet_manager.cc:1918] T c360b3942d00499880b2bfa0b7bb4ea8 P 16b0d7dcc332467fb0bfad2531110705: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 20:08:43.171633 7800 log.cc:1196] T c360b3942d00499880b2bfa0b7bb4ea8 P 16b0d7dcc332467fb0bfad2531110705: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-1/wal/wals/c360b3942d00499880b2bfa0b7bb4ea8
I20260620 20:08:43.171887 7800 ts_tablet_manager.cc:1939] T c360b3942d00499880b2bfa0b7bb4ea8 P 16b0d7dcc332467fb0bfad2531110705: Deleting consensus metadata
I20260620 20:08:43.173718 7810 tablet_bootstrap.cc:492] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705: Bootstrap starting.
I20260620 20:08:43.174746 7810 tablet_bootstrap.cc:654] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:43.175949 7810 tablet_bootstrap.cc:492] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705: No bootstrap required, opened a new log
I20260620 20:08:43.176038 7810 ts_tablet_manager.cc:1397] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260620 20:08:43.176570 7810 raft_consensus.cc:348] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:43.176680 7810 raft_consensus.cc:374] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:43.176713 7810 raft_consensus.cc:729] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16b0d7dcc332467fb0bfad2531110705, State: Initialized, Role: FOLLOWER
I20260620 20:08:43.176853 7810 consensus_queue.cc:260] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:43.177112 7810 ts_tablet_manager.cc:1428] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 20:08:43.198621 7631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 687984ab33f14923b56b4207159078e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:43.202706 7351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 687984ab33f14923b56b4207159078e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:43.202857 7798 ts_tablet_manager.cc:1918] T c360b3942d00499880b2bfa0b7bb4ea8 P f390f74c2612442c903fc4ddca73be71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 20:08:43.202994 7798 log.cc:1196] T c360b3942d00499880b2bfa0b7bb4ea8 P f390f74c2612442c903fc4ddca73be71: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-0/wal/wals/c360b3942d00499880b2bfa0b7bb4ea8
I20260620 20:08:43.203305 7798 ts_tablet_manager.cc:1939] T c360b3942d00499880b2bfa0b7bb4ea8 P f390f74c2612442c903fc4ddca73be71: Deleting consensus metadata
I20260620 20:08:43.203773 7201 catalog_manager.cc:4841] TS 16b0d7dcc332467fb0bfad2531110705 (127.5.91.130:41225): tablet c360b3942d00499880b2bfa0b7bb4ea8 (table test-table [id=3be58799487b4d4a8f03eca4fd5558a2]) successfully deleted
I20260620 20:08:43.205459 7811 tablet_bootstrap.cc:492] T 687984ab33f14923b56b4207159078e9 P f390f74c2612442c903fc4ddca73be71: Bootstrap starting.
I20260620 20:08:43.206566 7811 tablet_bootstrap.cc:654] T 687984ab33f14923b56b4207159078e9 P f390f74c2612442c903fc4ddca73be71: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:43.208094 7811 tablet_bootstrap.cc:492] T 687984ab33f14923b56b4207159078e9 P f390f74c2612442c903fc4ddca73be71: No bootstrap required, opened a new log
I20260620 20:08:43.208218 7811 ts_tablet_manager.cc:1397] T 687984ab33f14923b56b4207159078e9 P f390f74c2612442c903fc4ddca73be71: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 20:08:43.208626 7811 raft_consensus.cc:348] T 687984ab33f14923b56b4207159078e9 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:43.208761 7811 raft_consensus.cc:374] T 687984ab33f14923b56b4207159078e9 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:43.208787 7811 raft_consensus.cc:729] T 687984ab33f14923b56b4207159078e9 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f390f74c2612442c903fc4ddca73be71, State: Initialized, Role: FOLLOWER
I20260620 20:08:43.208905 7811 consensus_queue.cc:260] T 687984ab33f14923b56b4207159078e9 P f390f74c2612442c903fc4ddca73be71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:43.209174 7811 ts_tablet_manager.cc:1428] T 687984ab33f14923b56b4207159078e9 P f390f74c2612442c903fc4ddca73be71: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 20:08:43.210718 7799 ts_tablet_manager.cc:1918] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 20:08:43.211071 7799 log.cc:1196] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/c360b3942d00499880b2bfa0b7bb4ea8
I20260620 20:08:43.211649 7799 ts_tablet_manager.cc:1939] T c360b3942d00499880b2bfa0b7bb4ea8 P 2e5414be5a3c427595e36ef50951dc46: Deleting consensus metadata
I20260620 20:08:43.212064 7199 catalog_manager.cc:4841] TS f390f74c2612442c903fc4ddca73be71 (127.5.91.129:40759): tablet c360b3942d00499880b2bfa0b7bb4ea8 (table test-table [id=3be58799487b4d4a8f03eca4fd5558a2]) successfully deleted
I20260620 20:08:43.213384 7809 tablet_bootstrap.cc:492] T 687984ab33f14923b56b4207159078e9 P 2e5414be5a3c427595e36ef50951dc46: Bootstrap starting.
I20260620 20:08:43.214807 7809 tablet_bootstrap.cc:654] T 687984ab33f14923b56b4207159078e9 P 2e5414be5a3c427595e36ef50951dc46: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:43.216957 7809 tablet_bootstrap.cc:492] T 687984ab33f14923b56b4207159078e9 P 2e5414be5a3c427595e36ef50951dc46: No bootstrap required, opened a new log
I20260620 20:08:43.217062 7809 ts_tablet_manager.cc:1397] T 687984ab33f14923b56b4207159078e9 P 2e5414be5a3c427595e36ef50951dc46: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260620 20:08:43.217506 7809 raft_consensus.cc:348] T 687984ab33f14923b56b4207159078e9 P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:43.217629 7809 raft_consensus.cc:374] T 687984ab33f14923b56b4207159078e9 P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:43.217664 7809 raft_consensus.cc:729] T 687984ab33f14923b56b4207159078e9 P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e5414be5a3c427595e36ef50951dc46, State: Initialized, Role: FOLLOWER
I20260620 20:08:43.217795 7809 consensus_queue.cc:260] T 687984ab33f14923b56b4207159078e9 P 2e5414be5a3c427595e36ef50951dc46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:43.218075 7809 ts_tablet_manager.cc:1428] T 687984ab33f14923b56b4207159078e9 P 2e5414be5a3c427595e36ef50951dc46: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 20:08:43.218544 7816 raft_consensus.cc:482] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:08:43.218840 7816 raft_consensus.cc:504] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:43.220578 7816 leader_election.cc:288] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f390f74c2612442c903fc4ddca73be71 (127.5.91.129:40759), 2e5414be5a3c427595e36ef50951dc46 (127.5.91.131:43713)
I20260620 20:08:43.224924 7820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:43.221318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:44825 (local address 127.5.91.131:43713)
0620 20:08:43.221424 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:43.221428 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:43.221599 (+ 171us) server_negotiation.cc:407] Connection header received
0620 20:08:43.221758 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:43.221762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:43.221807 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:43.221879 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:43.222283 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:43.222788 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:43.223876 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:43.224282 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:43.224392 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:43.224532 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:43.224729 (+ 197us) server_negotiation.cc:299] Negotiation successful
0620 20:08:43.224800 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 20:08:43.224975 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:43.220918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:43713 (local address 127.5.91.130:44825)
0620 20:08:43.221504 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:43.221516 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:43.221657 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:43.221894 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:43.221897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:43.221936 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:43.222073 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:43.222080 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:43.222917 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:43.222920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:43.223764 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:43.223772 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:43.224392 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:43.224395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:43.224594 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:43.224619 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 20:08:43.224703 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 20:08:43.224778 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"mutex_wait_us":21,"thread_start_us":128,"threads_started":1}
I20260620 20:08:43.225004 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:43.221077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:41925 (local address 127.5.91.129:40759)
0620 20:08:43.221312 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:43.221325 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:43.221339 (+ 14us) server_negotiation.cc:407] Connection header received
0620 20:08:43.221444 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:43.221447 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:43.221492 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:43.221564 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:43.221822 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:43.222442 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:43.223519 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:43.224211 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:43.224576 (+ 365us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:43.224633 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:43.224779 (+ 146us) server_negotiation.cc:299] Negotiation successful
0620 20:08:43.224874 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":73,"threads_started":1}
I20260620 20:08:43.225272 7830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:43.220757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:40759 (local address 127.5.91.130:41925)
0620 20:08:43.221215 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:43.221230 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:43.221335 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:43.221585 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:43.221587 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:43.221603 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:43.221728 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:43.221733 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:43.222539 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:43.222542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:43.223408 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:43.223417 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:43.224547 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:43.224551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:43.224660 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:43.224682 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 20:08:43.224757 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 20:08:43.224839 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":20,"spinlock_wait_cycles":20352,"thread_start_us":59,"threads_started":1}
I20260620 20:08:43.225389 7650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "687984ab33f14923b56b4207159078e9" candidate_uuid: "16b0d7dcc332467fb0bfad2531110705" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e5414be5a3c427595e36ef50951dc46" is_pre_election: true
I20260620 20:08:43.225545 7650 raft_consensus.cc:2454] T 687984ab33f14923b56b4207159078e9 P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16b0d7dcc332467fb0bfad2531110705 in term 0.
I20260620 20:08:43.225919 7444 leader_election.cc:302] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16b0d7dcc332467fb0bfad2531110705, 2e5414be5a3c427595e36ef50951dc46; no voters:
I20260620 20:08:43.226007 7371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "687984ab33f14923b56b4207159078e9" candidate_uuid: "16b0d7dcc332467fb0bfad2531110705" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f390f74c2612442c903fc4ddca73be71" is_pre_election: true
I20260620 20:08:43.226184 7371 raft_consensus.cc:2454] T 687984ab33f14923b56b4207159078e9 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16b0d7dcc332467fb0bfad2531110705 in term 0.
I20260620 20:08:43.226223 7816 raft_consensus.cc:2781] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:08:43.226272 7816 raft_consensus.cc:482] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:08:43.226307 7816 raft_consensus.cc:3037] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:43.270823 7200 catalog_manager.cc:4841] TS 2e5414be5a3c427595e36ef50951dc46 (127.5.91.131:43713): tablet c360b3942d00499880b2bfa0b7bb4ea8 (table test-table [id=3be58799487b4d4a8f03eca4fd5558a2]) successfully deleted
I20260620 20:08:43.295645 7816 raft_consensus.cc:504] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:43.296335 7816 leader_election.cc:288] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705 [CANDIDATE]: Term 1 election: Requested vote from peers f390f74c2612442c903fc4ddca73be71 (127.5.91.129:40759), 2e5414be5a3c427595e36ef50951dc46 (127.5.91.131:43713)
I20260620 20:08:43.296808 7371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "687984ab33f14923b56b4207159078e9" candidate_uuid: "16b0d7dcc332467fb0bfad2531110705" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f390f74c2612442c903fc4ddca73be71"
I20260620 20:08:43.296815 7650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "687984ab33f14923b56b4207159078e9" candidate_uuid: "16b0d7dcc332467fb0bfad2531110705" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e5414be5a3c427595e36ef50951dc46"
I20260620 20:08:43.296958 7371 raft_consensus.cc:3037] T 687984ab33f14923b56b4207159078e9 P f390f74c2612442c903fc4ddca73be71 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:43.296967 7650 raft_consensus.cc:3037] T 687984ab33f14923b56b4207159078e9 P 2e5414be5a3c427595e36ef50951dc46 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:43.298010 7371 raft_consensus.cc:2454] T 687984ab33f14923b56b4207159078e9 P f390f74c2612442c903fc4ddca73be71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16b0d7dcc332467fb0bfad2531110705 in term 1.
I20260620 20:08:43.298205 7650 raft_consensus.cc:2454] T 687984ab33f14923b56b4207159078e9 P 2e5414be5a3c427595e36ef50951dc46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16b0d7dcc332467fb0bfad2531110705 in term 1.
I20260620 20:08:43.298414 7443 leader_election.cc:302] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16b0d7dcc332467fb0bfad2531110705, f390f74c2612442c903fc4ddca73be71; no voters:
I20260620 20:08:43.298623 7816 raft_consensus.cc:2781] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:43.298887 7816 raft_consensus.cc:686] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705 [term 1 LEADER]: Becoming Leader. State: Replica: 16b0d7dcc332467fb0bfad2531110705, State: Running, Role: LEADER
I20260620 20:08:43.299167 7816 consensus_queue.cc:237] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } }
I20260620 20:08:43.302402 7214 catalog_manager.cc:5495] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16b0d7dcc332467fb0bfad2531110705 (127.5.91.130). New cstate: current_term: 1 leader_uuid: "16b0d7dcc332467fb0bfad2531110705" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16b0d7dcc332467fb0bfad2531110705" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 } health_report { overall_health: UNKNOWN } } }
I20260620 20:08:43.341535 7834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:43.337183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36942 (local address 127.5.91.130:41225)
0620 20:08:43.337470 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:43.337476 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:43.337486 (+ 10us) server_negotiation.cc:407] Connection header received
0620 20:08:43.337554 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:43.337556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:43.337607 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:43.337803 (+ 196us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:08:43.338385 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:43.339162 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:43.340070 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:43.340301 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:43.340386 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:43.340643 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:08:43.341123 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:08:43.341222 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:43.341288 (+ 66us) server_negotiation.cc:299] Negotiation successful
0620 20:08:43.341346 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":114,"threads_started":1}
I20260620 20:08:43.354333 7837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:43.350487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:40759 (local address 127.5.91.130:49305)
0620 20:08:43.350917 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:43.350934 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:43.351042 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:43.351353 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:43.351355 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:43.351363 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 20:08:43.351484 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:43.351493 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:43.352292 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:43.352295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:43.352951 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:43.352961 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:43.353442 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:43.353445 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:43.353558 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:43.353608 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 20:08:43.353951 (+ 343us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 20:08:43.353959 (+ 8us) client_negotiation.cc:769] Sending connection context
0620 20:08:43.354029 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 20:08:43.354108 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":96,"threads_started":1}
I20260620 20:08:43.354460 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:43.351002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:49305 (local address 127.5.91.129:40759)
0620 20:08:43.351154 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:43.351158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:43.351170 (+ 12us) server_negotiation.cc:407] Connection header received
0620 20:08:43.351217 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:43.351219 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:43.351263 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:43.351329 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:08:43.351614 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:43.352180 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:43.353080 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:43.353307 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:43.353408 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:43.353755 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:08:43.353862 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:08:43.353921 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:43.354205 (+ 284us) server_negotiation.cc:299] Negotiation successful
0620 20:08:43.354306 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260620 20:08:43.362102 7371 raft_consensus.cc:1261] T 687984ab33f14923b56b4207159078e9 P f390f74c2612442c903fc4ddca73be71 [term 1 FOLLOWER]: Refusing update from remote peer 16b0d7dcc332467fb0bfad2531110705: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:08:43.362112 7650 raft_consensus.cc:1261] T 687984ab33f14923b56b4207159078e9 P 2e5414be5a3c427595e36ef50951dc46 [term 1 FOLLOWER]: Refusing update from remote peer 16b0d7dcc332467fb0bfad2531110705: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:08:43.362632 7816 consensus_queue.cc:1035] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f390f74c2612442c903fc4ddca73be71" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:08:43.362825 7832 consensus_queue.cc:1035] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e5414be5a3c427595e36ef50951dc46" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 43713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:08:43.382076 7834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:43.377615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:46481 (local address 127.5.91.130:41225)
0620 20:08:43.378330 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:43.378333 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:43.378345 (+ 12us) server_negotiation.cc:407] Connection header received
0620 20:08:43.378385 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:43.378387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:43.378435 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:43.378496 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:08:43.378748 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:43.379743 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:43.380986 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:43.381316 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:43.381400 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:43.381616 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:08:43.381692 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:08:43.381751 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:43.381879 (+ 128us) server_negotiation.cc:299] Negotiation successful
0620 20:08:43.381972 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641}
I20260620 20:08:43.382089 7842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:43.377562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:41225 (local address 127.5.91.129:46481)
0620 20:08:43.377860 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:43.377875 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:43.377963 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:43.378512 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:43.378515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:43.378523 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 20:08:43.378636 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:43.378642 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:43.379862 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:43.379865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:43.380802 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:43.380810 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:43.381426 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:43.381428 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:43.381500 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:43.381528 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 20:08:43.381793 (+ 265us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 20:08:43.381799 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 20:08:43.381894 (+ 95us) client_negotiation.cc:240] Negotiation successful
0620 20:08:43.381965 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":103,"threads_started":1}
W20260620 20:08:43.382961 7782 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260620 20:08:44.951773 7211 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:08:44.951964 7211 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:08:44.955050 7211 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f7d11bf80d7548788bed233a0d778122: Sending DeleteTablet for 3 replicas of tablet 687984ab33f14923b56b4207159078e9
I20260620 20:08:44.955608 7351 tablet_service.cc:1514] Processing DeleteTablet for tablet 687984ab33f14923b56b4207159078e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:44 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:45938
I20260620 20:08:44.955608 7491 tablet_service.cc:1514] Processing DeleteTablet for tablet 687984ab33f14923b56b4207159078e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:44 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:58318
I20260620 20:08:44.955715 7631 tablet_service.cc:1514] Processing DeleteTablet for tablet 687984ab33f14923b56b4207159078e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:44 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:42288
I20260620 20:08:44.955946 7866 tablet_replica.cc:330] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705: stopping tablet replica
I20260620 20:08:44.956071 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 7286
I20260620 20:08:44.956091 7866 raft_consensus.cc:2229] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705 [term 1 LEADER]: Raft consensus shutting down.
I20260620 20:08:44.958279 7866 raft_consensus.cc:2258] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:08:44.958587 7867 tablet_replica.cc:330] T 687984ab33f14923b56b4207159078e9 P 2e5414be5a3c427595e36ef50951dc46: stopping tablet replica
I20260620 20:08:44.958732 7867 raft_consensus.cc:2229] T 687984ab33f14923b56b4207159078e9 P 2e5414be5a3c427595e36ef50951dc46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:08:44.958873 7867 raft_consensus.cc:2258] T 687984ab33f14923b56b4207159078e9 P 2e5414be5a3c427595e36ef50951dc46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:08:44.958971 7866 ts_tablet_manager.cc:1905] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:08:44.959458 7867 ts_tablet_manager.cc:1905] T 687984ab33f14923b56b4207159078e9 P 2e5414be5a3c427595e36ef50951dc46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:08:44.961774 7866 ts_tablet_manager.cc:1918] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 20:08:44.961869 7866 log.cc:1196] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-1/wal/wals/687984ab33f14923b56b4207159078e9
I20260620 20:08:44.961911 7867 ts_tablet_manager.cc:1918] T 687984ab33f14923b56b4207159078e9 P 2e5414be5a3c427595e36ef50951dc46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 20:08:44.962013 7867 log.cc:1196] T 687984ab33f14923b56b4207159078e9 P 2e5414be5a3c427595e36ef50951dc46: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/687984ab33f14923b56b4207159078e9
I20260620 20:08:44.962179 7866 ts_tablet_manager.cc:1939] T 687984ab33f14923b56b4207159078e9 P 16b0d7dcc332467fb0bfad2531110705: Deleting consensus metadata
I20260620 20:08:44.962268 7867 ts_tablet_manager.cc:1939] T 687984ab33f14923b56b4207159078e9 P 2e5414be5a3c427595e36ef50951dc46: Deleting consensus metadata
I20260620 20:08:44.963212 7201 catalog_manager.cc:4841] TS 16b0d7dcc332467fb0bfad2531110705 (127.5.91.130:41225): tablet 687984ab33f14923b56b4207159078e9 (table test-table [id=3bac536ab248439e9e86118ee2562a94]) successfully deleted
W20260620 20:08:44.963315 7199 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.91.129:40759 (error 108)
I20260620 20:08:44.963678 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 7426
I20260620 20:08:44.965912 7200 catalog_manager.cc:4841] TS 2e5414be5a3c427595e36ef50951dc46 (127.5.91.131:43713): tablet 687984ab33f14923b56b4207159078e9 (table test-table [id=3bac536ab248439e9e86118ee2562a94]) successfully deleted
I20260620 20:08:44.966082 7869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:44.964778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:40759 (local address 127.0.0.1:42636)
0620 20:08:44.965878 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:44.965974 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.129:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1012,"thread_start_us":138,"threads_started":1}
W20260620 20:08:44.966259 7199 catalog_manager.cc:4584] TS f390f74c2612442c903fc4ddca73be71 (127.5.91.129:40759): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 687984ab33f14923b56b4207159078e9: Network error: Client connection negotiation failed: client connection to 127.5.91.129:40759: connect: Connection refused (error 111)
I20260620 20:08:44.969976 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 7566
I20260620 20:08:44.977099 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 7182
2026-06-20T20:08:44Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6473 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:08:45 dist-test-slave-l8h4 krb5kdc[7872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:08:45 dist-test-slave-l8h4 krb5kdc[7872](info): set up 2 sockets
Jun 20 20:08:45 dist-test-slave-l8h4 krb5kdc[7872](info): commencing operation
krb5kdc: starting...
W20260620 20:08:45.975358 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.975s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:08:46 dist-test-slave-l8h4 krb5kdc[7872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:08:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:08:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:08:46.114342 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:46181
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:46181
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:46.235028 7885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:46.235312 7885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:46.235355 7885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:46.240302 7885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:08:46.240370 7885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:46.240389 7885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:46.240406 7885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:08:46.240492 7885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:08:46.246389 7885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:46181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:46181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:46.247524 7885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:46.248656 7885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:46.260390 7891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:46.260449 7890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:46.260610 7885 server_base.cc:1034] running on GCE node
W20260620 20:08:46.260390 7893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:46.261291 7885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:46.262514 7885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:46.263793 7885 hybrid_clock.cc:648] HybridClock initialized: now 1781986126263644 us; error 74 us; skew 500 ppm
Jun 20 20:08:46 dist-test-slave-l8h4 krb5kdc[7872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986126, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:46.272217 7885 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:08:46.273309 7885 webserver.cc:460] Webserver started at http://127.5.91.190:39221/ using document root <none> and password file <none>
I20260620 20:08:46.273919 7885 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:46.273984 7885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:46.274183 7885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:46.276321 7885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "3055d45826464a2ca619423d08204856"
format_stamp: "Formatted at 2026-06-20 20:08:46 on dist-test-slave-l8h4"
server_key: "05a7eef48cc17ede33b02c8cd82fc5c9"
server_key_iv: "1a8023a055eb0f9bc07857f135b61af8"
server_key_version: "encryptionkey@0"
I20260620 20:08:46.276849 7885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "3055d45826464a2ca619423d08204856"
format_stamp: "Formatted at 2026-06-20 20:08:46 on dist-test-slave-l8h4"
server_key: "05a7eef48cc17ede33b02c8cd82fc5c9"
server_key_iv: "1a8023a055eb0f9bc07857f135b61af8"
server_key_version: "encryptionkey@0"
I20260620 20:08:46.281131 7885 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260620 20:08:46.284650 7900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:46.286263 7885 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:08:46.286410 7885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "3055d45826464a2ca619423d08204856"
format_stamp: "Formatted at 2026-06-20 20:08:46 on dist-test-slave-l8h4"
server_key: "05a7eef48cc17ede33b02c8cd82fc5c9"
server_key_iv: "1a8023a055eb0f9bc07857f135b61af8"
server_key_version: "encryptionkey@0"
I20260620 20:08:46.286522 7885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:46.300065 7885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:46.303263 7885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:46.303422 7885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:46.310997 7885 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:46181
I20260620 20:08:46.311177 7952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:46181 every 8 connection(s)
I20260620 20:08:46.312038 7885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:08:46.315718 7953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:46.321864 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 7885
I20260620 20:08:46.321996 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:08:46.322377 5486 external_mini_cluster.cc:1428] Setting key 2f8dc4dea6eb54f4199a06a6f205efe3
I20260620 20:08:46.323648 7953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856: Bootstrap starting.
I20260620 20:08:46.326241 7953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:46.327072 7953 log.cc:824] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:46.329928 7953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856: No bootstrap required, opened a new log
Jun 20 20:08:46 dist-test-slave-l8h4 krb5kdc[7872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:46.335276 7953 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3055d45826464a2ca619423d08204856" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46181 } }
I20260620 20:08:46.335659 7953 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:46.335798 7953 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3055d45826464a2ca619423d08204856, State: Initialized, Role: FOLLOWER
I20260620 20:08:46.336387 7953 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3055d45826464a2ca619423d08204856" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46181 } }
I20260620 20:08:46.336565 7953 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:08:46.336611 7953 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:08:46.336694 7953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:46.337932 7953 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3055d45826464a2ca619423d08204856" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46181 } }
I20260620 20:08:46.338307 7953 leader_election.cc:302] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3055d45826464a2ca619423d08204856; no voters:
I20260620 20:08:46.338611 7953 leader_election.cc:288] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:08:46.338738 7958 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:46.339103 7958 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856 [term 1 LEADER]: Becoming Leader. State: Replica: 3055d45826464a2ca619423d08204856, State: Running, Role: LEADER
I20260620 20:08:46.339510 7958 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3055d45826464a2ca619423d08204856" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46181 } }
I20260620 20:08:46.339883 7953 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:08:46.340927 7958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3055d45826464a2ca619423d08204856. Latest consensus state: current_term: 1 leader_uuid: "3055d45826464a2ca619423d08204856" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3055d45826464a2ca619423d08204856" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46181 } } }
I20260620 20:08:46.341039 7956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:46.324277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52878 (local address 127.5.91.190:46181)
0620 20:08:46.324812 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:46.324835 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:46.324877 (+ 42us) server_negotiation.cc:407] Connection header received
0620 20:08:46.326044 (+ 1167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:46.326067 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:46.326521 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:46.326834 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:46.327722 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:46.328778 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:46.330072 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:46.330629 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:46.330761 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:46.334081 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:46.334086 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:46.334100 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:46.334129 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:46.336463 (+ 2334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:46.337187 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:46.337194 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:46.337199 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:46.337373 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:46.337834 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:46.337838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:46.337839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:46.338618 (+ 779us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:46.338870 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:46.339711 (+ 841us) server_negotiation.cc:299] Negotiation successful
0620 20:08:46.340032 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":162,"threads_started":1}
I20260620 20:08:46.341038 7958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856 [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:46.341301 7960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3055d45826464a2ca619423d08204856" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3055d45826464a2ca619423d08204856" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46181 } } }
I20260620 20:08:46.341396 7960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856 [sys.catalog]: This master's current role is: LEADER
W20260620 20:08:46.344558 7973 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 20:08:46.344623 7973 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 20:08:46.344704 7968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:08:46.345398 7968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:08:46.350579 7968 catalog_manager.cc:1347] Generated new cluster ID: 71982c9042ad417faa14429728592950
I20260620 20:08:46.350652 7968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:08:46.363795 7968 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:08:46.365127 7968 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:08:46.373571 7968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3055d45826464a2ca619423d08204856: Generated new TSK 0
I20260620 20:08:46.374302 7968 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.91.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
I20260620 20:08:46.405902 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46181
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:46.531576 7979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:46.531834 7979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:46.531883 7979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:46.535876 7979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:46.535946 7979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:46.536006 7979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:08:46.542704 7979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:46.544054 7979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:46.545331 7979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:46.556239 7984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:46.556242 7985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:46.556242 7987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:46.556617 7979 server_base.cc:1034] running on GCE node
I20260620 20:08:46.557257 7979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:46.558066 7979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:46.559281 7979 hybrid_clock.cc:648] HybridClock initialized: now 1781986126559254 us; error 53 us; skew 500 ppm
Jun 20 20:08:46 dist-test-slave-l8h4 krb5kdc[7872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986126, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:46.562577 7979 init.cc:377] Logged in from keytab as kudu/127.5.91.129@KRBTEST.COM (short username kudu)
I20260620 20:08:46.563483 7979 webserver.cc:460] Webserver started at http://127.5.91.129:35823/ using document root <none> and password file <none>
I20260620 20:08:46.564025 7979 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:46.564083 7979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:46.564272 7979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:46.566354 7979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "20c41e3cb6134e96a3b4e1f2abf42ad8"
format_stamp: "Formatted at 2026-06-20 20:08:46 on dist-test-slave-l8h4"
server_key: "5b56349347552749ff983082d7c67877"
server_key_iv: "b449ca96e513537cb387930f9885ed06"
server_key_version: "encryptionkey@0"
I20260620 20:08:46.566887 7979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "20c41e3cb6134e96a3b4e1f2abf42ad8"
format_stamp: "Formatted at 2026-06-20 20:08:46 on dist-test-slave-l8h4"
server_key: "5b56349347552749ff983082d7c67877"
server_key_iv: "b449ca96e513537cb387930f9885ed06"
server_key_version: "encryptionkey@0"
I20260620 20:08:46.572326 7979 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 20:08:46.575296 7994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:46.576655 7979 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 20:08:46.576871 7979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "20c41e3cb6134e96a3b4e1f2abf42ad8"
format_stamp: "Formatted at 2026-06-20 20:08:46 on dist-test-slave-l8h4"
server_key: "5b56349347552749ff983082d7c67877"
server_key_iv: "b449ca96e513537cb387930f9885ed06"
server_key_version: "encryptionkey@0"
I20260620 20:08:46.577383 7979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:46.596202 7979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:46.601910 7979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:46.602128 7979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:46.602803 7979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:08:46.603665 7979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:46.603732 7979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:46.603788 7979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:46.603817 7979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:46.616428 7979 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:45521
I20260620 20:08:46.616492 8107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:45521 every 8 connection(s)
I20260620 20:08:46.617900 7979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:08:46.623435 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 7979
I20260620 20:08:46.624037 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:08:46.624574 5486 external_mini_cluster.cc:1428] Setting key 717c1eb96d7f0d63d5b21aa8fdec525d
Jun 20 20:08:46 dist-test-slave-l8h4 krb5kdc[7872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986126, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.91.130@KRBTEST.COM; defaulting to no policy
I20260620 20:08:46.638813 7956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:46.620761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:34175 (local address 127.5.91.190:46181)
0620 20:08:46.621104 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:46.621108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:46.622660 (+ 1552us) server_negotiation.cc:407] Connection header received
0620 20:08:46.624078 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:46.624083 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:46.624144 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:46.624469 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:46.625970 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:46.626773 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:46.628187 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:46.628585 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:46.628768 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:46.633577 (+ 4809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:46.633590 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:46.633593 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:46.633636 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:46.635263 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:46.636225 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:46.636230 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:46.636434 (+ 204us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:46.636559 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:46.637234 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:46.637238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:46.637240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:46.637524 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:46.637628 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:46.638264 (+ 636us) server_negotiation.cc:299] Negotiation successful
0620 20:08:46.638419 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.130@KRBTEST.COM" created.
I20260620 20:08:46.640893 8110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:46.621349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46181 (local address 127.5.91.129:34175)
0620 20:08:46.622457 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:46.622508 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:46.623829 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:46.624630 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:46.624638 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:46.625139 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:46.625748 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:46.625763 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:46.626993 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:46.626996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:46.627949 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:46.627960 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:46.628901 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:46.628906 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:46.629295 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:46.630605 (+ 1310us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:46.630632 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:46.633247 (+ 2615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:46.635505 (+ 2258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:46.635572 (+ 67us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:46.635602 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:46.636028 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:46.636696 (+ 668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:46.636699 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:46.636701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:46.637055 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:46.637655 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:46.637663 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:46.637996 (+ 333us) client_negotiation.cc:769] Sending connection context
0620 20:08:46.638223 (+ 227us) client_negotiation.cc:240] Negotiation successful
0620 20:08:46.638916 (+ 693us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":876,"thread_start_us":617,"threads_started":1}
I20260620 20:08:46.643360 8108 heartbeater.cc:347] Connected to a master server at 127.5.91.190:46181
I20260620 20:08:46.643883 8108 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:46.644886 8108 heartbeater.cc:511] Master 127.5.91.190:46181 requested a full tablet report, sending...
I20260620 20:08:46.646780 7917 ts_manager.cc:194] Registered new tserver with Master: 20c41e3cb6134e96a3b4e1f2abf42ad8 (127.5.91.129:45521)
I20260620 20:08:46.648005 7917 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:34175
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
I20260620 20:08:46.667904 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46181
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:46.812114 8113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:46.812408 8113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:46.812647 8113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:46.817948 8113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:46.818094 8113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:46.818243 8113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:08:46.825853 8113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:46.827452 8113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:46.828526 8113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:46.838330 8121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:46.838342 8118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:46.838438 8119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:46.838989 8113 server_base.cc:1034] running on GCE node
I20260620 20:08:46.839427 8113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:46.840478 8113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:46.841815 8113 hybrid_clock.cc:648] HybridClock initialized: now 1781986126841784 us; error 38 us; skew 500 ppm
Jun 20 20:08:46 dist-test-slave-l8h4 krb5kdc[7872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986126, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:46.845168 8113 init.cc:377] Logged in from keytab as kudu/127.5.91.130@KRBTEST.COM (short username kudu)
I20260620 20:08:46.846465 8113 webserver.cc:460] Webserver started at http://127.5.91.130:33677/ using document root <none> and password file <none>
I20260620 20:08:46.847034 8113 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:46.847105 8113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:46.847295 8113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:46.849411 8113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "9188d72fcf4641fabdeae39143534cae"
format_stamp: "Formatted at 2026-06-20 20:08:46 on dist-test-slave-l8h4"
server_key: "e940be91b666eeeef427d2fdbd115522"
server_key_iv: "f7600f14519ce3bf30977b072906467e"
server_key_version: "encryptionkey@0"
I20260620 20:08:46.849978 8113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "9188d72fcf4641fabdeae39143534cae"
format_stamp: "Formatted at 2026-06-20 20:08:46 on dist-test-slave-l8h4"
server_key: "e940be91b666eeeef427d2fdbd115522"
server_key_iv: "f7600f14519ce3bf30977b072906467e"
server_key_version: "encryptionkey@0"
I20260620 20:08:46.854350 8113 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260620 20:08:46.858568 8128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:46.860131 8113 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 20:08:46.860260 8113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "9188d72fcf4641fabdeae39143534cae"
format_stamp: "Formatted at 2026-06-20 20:08:46 on dist-test-slave-l8h4"
server_key: "e940be91b666eeeef427d2fdbd115522"
server_key_iv: "f7600f14519ce3bf30977b072906467e"
server_key_version: "encryptionkey@0"
I20260620 20:08:46.860381 8113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:46.872839 8113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:46.877628 8113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:46.877831 8113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:46.878513 8113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:08:46.879371 8113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:46.879419 8113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:46.879477 8113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:46.879506 8113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:46.890458 8113 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:42343
I20260620 20:08:46.890547 8241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:42343 every 8 connection(s)
I20260620 20:08:46.891777 8113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:08:46.896219 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 8113
I20260620 20:08:46.896358 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:08:46.896656 5486 external_mini_cluster.cc:1428] Setting key c36a94bb9c4cc4c4de0df8d7973b7f08
Jun 20 20:08:46 dist-test-slave-l8h4 krb5kdc[7872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986126, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.91.131@KRBTEST.COM; defaulting to no policy
I20260620 20:08:46.909262 7956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:46.894474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:38599 (local address 127.5.91.190:46181)
0620 20:08:46.894730 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:46.894734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:46.896084 (+ 1350us) server_negotiation.cc:407] Connection header received
0620 20:08:46.897722 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:46.897725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:46.897771 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:46.897861 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:46.899174 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:46.899915 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:46.901604 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:46.901824 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:46.901915 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:46.905057 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:46.905061 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:46.905063 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:46.905092 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:46.906782 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:46.907449 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:46.907453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:46.907455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:46.907542 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:46.908034 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:46.908038 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:46.908039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:46.908256 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:46.908340 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:46.908995 (+ 655us) server_negotiation.cc:299] Negotiation successful
0620 20:08:46.909110 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.131@KRBTEST.COM" created.
I20260620 20:08:46.910336 8244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:46.895176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46181 (local address 127.5.91.130:38599)
0620 20:08:46.895847 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:46.895905 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:46.897391 (+ 1486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:46.898088 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:46.898096 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:46.898504 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:46.899016 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:46.899030 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:46.900064 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:46.900068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:46.901468 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:46.901481 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:46.901979 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:46.901982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:46.902213 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:46.902909 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:46.902930 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:46.904772 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:46.906962 (+ 2190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:46.906969 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:46.906991 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:46.907329 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:46.907692 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:46.907695 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:46.907697 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:46.907891 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:46.908392 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:46.908399 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:46.908736 (+ 337us) client_negotiation.cc:769] Sending connection context
0620 20:08:46.908921 (+ 185us) client_negotiation.cc:240] Negotiation successful
0620 20:08:46.909245 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":185,"threads_started":1}
I20260620 20:08:46.912468 8242 heartbeater.cc:347] Connected to a master server at 127.5.91.190:46181
I20260620 20:08:46.912788 8242 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:46.913353 8242 heartbeater.cc:511] Master 127.5.91.190:46181 requested a full tablet report, sending...
I20260620 20:08:46.915182 7917 ts_manager.cc:194] Registered new tserver with Master: 9188d72fcf4641fabdeae39143534cae (127.5.91.130:42343)
I20260620 20:08:46.915814 7917 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:38599
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
I20260620 20:08:46.935644 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46181
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:47.058326 8247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:47.058599 8247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:47.058642 8247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:47.062667 8247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:47.062724 8247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:47.062791 8247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:08:47.068159 8247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:47.069217 8247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:47.070377 8247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:47.080165 8253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:47.080227 8255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:47.080228 8252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:47.080461 8247 server_base.cc:1034] running on GCE node
I20260620 20:08:47.080870 8247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:47.081559 8247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:47.082741 8247 hybrid_clock.cc:648] HybridClock initialized: now 1781986127082723 us; error 30 us; skew 500 ppm
Jun 20 20:08:47 dist-test-slave-l8h4 krb5kdc[7872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986127, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:47.086232 8247 init.cc:377] Logged in from keytab as kudu/127.5.91.131@KRBTEST.COM (short username kudu)
I20260620 20:08:47.087160 8247 webserver.cc:460] Webserver started at http://127.5.91.131:35445/ using document root <none> and password file <none>
I20260620 20:08:47.087721 8247 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:47.087787 8247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:47.087981 8247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:47.090121 8247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "b598b31f854b4e429b01cb94e795e518"
format_stamp: "Formatted at 2026-06-20 20:08:47 on dist-test-slave-l8h4"
server_key: "9d704367ed799dd6bea427b1615e550e"
server_key_iv: "5923ef9f3f1d4f7909474f7873a24932"
server_key_version: "encryptionkey@0"
I20260620 20:08:47.090776 8247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "b598b31f854b4e429b01cb94e795e518"
format_stamp: "Formatted at 2026-06-20 20:08:47 on dist-test-slave-l8h4"
server_key: "9d704367ed799dd6bea427b1615e550e"
server_key_iv: "5923ef9f3f1d4f7909474f7873a24932"
server_key_version: "encryptionkey@0"
I20260620 20:08:47.095063 8247 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260620 20:08:47.098090 8262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:47.099858 8247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:08:47.100033 8247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "b598b31f854b4e429b01cb94e795e518"
format_stamp: "Formatted at 2026-06-20 20:08:47 on dist-test-slave-l8h4"
server_key: "9d704367ed799dd6bea427b1615e550e"
server_key_iv: "5923ef9f3f1d4f7909474f7873a24932"
server_key_version: "encryptionkey@0"
I20260620 20:08:47.100160 8247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:47.131486 8247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:47.134953 8247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:47.135149 8247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:47.135764 8247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:08:47.136850 8247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:47.137063 8247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:47.137230 8247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:47.137308 8247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:47.148357 8247 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:36119
I20260620 20:08:47.148377 8375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:36119 every 8 connection(s)
I20260620 20:08:47.149277 8247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:08:47.154358 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 8247
I20260620 20:08:47.154553 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:08:47.154925 5486 external_mini_cluster.cc:1428] Setting key b75a694dc753b7fc948e0d9b4b747f24
Jun 20 20:08:47 dist-test-slave-l8h4 krb5kdc[7872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986127, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:47.168107 7956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:47.152241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:43427 (local address 127.5.91.190:46181)
0620 20:08:47.152485 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:47.152490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:47.153759 (+ 1269us) server_negotiation.cc:407] Connection header received
0620 20:08:47.155758 (+ 1999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:47.155762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:47.155842 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:47.155926 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:47.157902 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:47.158676 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:47.160138 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:47.160387 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:47.160498 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:47.164023 (+ 3525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:47.164027 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:47.164029 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:47.164061 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:47.165882 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:47.166526 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:47.166529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:47.166531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:47.166606 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:47.166950 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:47.166952 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:47.166953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:47.167156 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:47.167241 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:47.167799 (+ 558us) server_negotiation.cc:299] Negotiation successful
0620 20:08:47.167940 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260620 20:08:47.169041 8378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:47.152635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46181 (local address 127.5.91.131:43427)
0620 20:08:47.153503 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:47.153573 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:47.155453 (+ 1880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:47.156195 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:47.156205 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:47.156967 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:47.157554 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:47.157575 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:47.158836 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:47.158840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:47.159962 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:47.159978 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:47.160652 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:47.160672 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:47.161003 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:47.161716 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:47.161735 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:47.163766 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:47.166050 (+ 2284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:47.166057 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:47.166072 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:47.166424 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:47.166694 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:47.166697 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:47.166699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:47.166849 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:47.167254 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:47.167262 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:47.167540 (+ 278us) client_negotiation.cc:769] Sending connection context
0620 20:08:47.167727 (+ 187us) client_negotiation.cc:240] Negotiation successful
0620 20:08:47.168090 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":159,"threads_started":1}
I20260620 20:08:47.170241 8376 heartbeater.cc:347] Connected to a master server at 127.5.91.190:46181
I20260620 20:08:47.170642 8376 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:47.171360 8376 heartbeater.cc:511] Master 127.5.91.190:46181 requested a full tablet report, sending...
I20260620 20:08:47.172902 7917 ts_manager.cc:194] Registered new tserver with Master: b598b31f854b4e429b01cb94e795e518 (127.5.91.131:36119)
I20260620 20:08:47.173516 7917 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:43427
I20260620 20:08:47.183796 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 20:08:47.197918 7956 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:08:47.195899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52880 (local address 127.5.91.190:46181)
0620 20:08:47.196023 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:47.196027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:47.196218 (+ 191us) server_negotiation.cc:407] Connection header received
0620 20:08:47.196289 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:47.196292 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:47.196345 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:47.196441 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:47.197679 (+ 1238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52880: BlockingRecv error: recv got EOF from 127.0.0.1:52880 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 20:08:47.199075 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 7979
I20260620 20:08:47.204350 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 8113
I20260620 20:08:47.209877 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 8247
I20260620 20:08:47.215013 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 7885
2026-06-20T20:08:47Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2236 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:08:47 dist-test-slave-l8h4 krb5kdc[8388](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:08:47 dist-test-slave-l8h4 krb5kdc[8388](info): set up 2 sockets
Jun 20 20:08:47 dist-test-slave-l8h4 krb5kdc[8388](info): commencing operation
krb5kdc: starting...
W20260620 20:08:48.231372 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.995s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:08:48 dist-test-slave-l8h4 krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:08:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:08:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:08:48.380091 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:37401
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:46749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:37401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:48.510258 8401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:48.510566 8401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:48.510625 8401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:48.514878 8401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:08:48.514954 8401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:48.514981 8401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:48.515000 8401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:08:48.515021 8401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:08:48.521840 8401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:46749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:37401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:37401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:48.523999 8401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:48.525455 8401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:48.536969 8407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:48.537103 8406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:48.537173 8409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:48.537696 8401 server_base.cc:1034] running on GCE node
I20260620 20:08:48.538724 8401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:48.540226 8401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:48.541530 8401 hybrid_clock.cc:648] HybridClock initialized: now 1781986128541502 us; error 70 us; skew 500 ppm
Jun 20 20:08:48 dist-test-slave-l8h4 krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986128, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:48.544725 8401 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:08:48.545764 8401 webserver.cc:460] Webserver started at http://127.5.91.190:33255/ using document root <none> and password file <none>
I20260620 20:08:48.546375 8401 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:48.546446 8401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:48.546653 8401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:48.549000 8401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "343807d90ff6423b8dfc8ce580727149"
format_stamp: "Formatted at 2026-06-20 20:08:48 on dist-test-slave-l8h4"
server_key: "950e44c07867bfe6709d910c16520ccd"
server_key_iv: "e58fecac8abe20095139868d518f179e"
server_key_version: "encryptionkey@0"
I20260620 20:08:48.549592 8401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "343807d90ff6423b8dfc8ce580727149"
format_stamp: "Formatted at 2026-06-20 20:08:48 on dist-test-slave-l8h4"
server_key: "950e44c07867bfe6709d910c16520ccd"
server_key_iv: "e58fecac8abe20095139868d518f179e"
server_key_version: "encryptionkey@0"
I20260620 20:08:48.554939 8401 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260620 20:08:48.558126 8416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:48.560045 8401 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 20:08:48.560204 8401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "343807d90ff6423b8dfc8ce580727149"
format_stamp: "Formatted at 2026-06-20 20:08:48 on dist-test-slave-l8h4"
server_key: "950e44c07867bfe6709d910c16520ccd"
server_key_iv: "e58fecac8abe20095139868d518f179e"
server_key_version: "encryptionkey@0"
I20260620 20:08:48.560329 8401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:48.582064 8401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:48.586719 8401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:48.586999 8401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:48.595295 8401 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:37401
I20260620 20:08:48.595801 8468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:37401 every 8 connection(s)
I20260620 20:08:48.596361 8401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:08:48.598443 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 8401
I20260620 20:08:48.598556 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:08:48.598814 5486 external_mini_cluster.cc:1428] Setting key bf246eea524d95cc5ab7bb263c7826e7
I20260620 20:08:48.600410 8469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:48.609243 8469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149: Bootstrap starting.
Jun 20 20:08:48 dist-test-slave-l8h4 krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:48.612514 8469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:48.613507 8469 log.cc:824] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:48.616472 8469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149: No bootstrap required, opened a new log
I20260620 20:08:48.618271 8472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:48.600361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46024 (local address 127.5.91.190:37401)
0620 20:08:48.601401 (+ 1040us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:48.601419 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:48.601462 (+ 43us) server_negotiation.cc:407] Connection header received
0620 20:08:48.602221 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:48.602245 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:48.602607 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:48.602964 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:48.603779 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:48.605178 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:48.606452 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:48.607117 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:48.607301 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:48.611475 (+ 4174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:48.611480 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:48.611494 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:48.611665 (+ 171us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:48.613828 (+ 2163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:48.614510 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:48.614555 (+ 45us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:48.614566 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:48.614840 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:48.615173 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:48.615177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:48.615179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:48.615727 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:48.615936 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:48.616243 (+ 307us) server_negotiation.cc:299] Negotiation successful
0620 20:08:48.616661 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":864,"thread_start_us":296,"threads_started":1}
I20260620 20:08:48.621927 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:08:48.622740 8469 raft_consensus.cc:348] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "343807d90ff6423b8dfc8ce580727149" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 37401 } }
I20260620 20:08:48.623085 8469 raft_consensus.cc:374] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:48.623134 8469 raft_consensus.cc:729] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343807d90ff6423b8dfc8ce580727149, State: Initialized, Role: FOLLOWER
I20260620 20:08:48.623620 8469 consensus_queue.cc:260] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "343807d90ff6423b8dfc8ce580727149" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 37401 } }
I20260620 20:08:48.623809 8469 raft_consensus.cc:388] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:08:48.623884 8469 raft_consensus.cc:482] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:08:48.623997 8469 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:48.642953 8472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:48.636897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46038 (local address 127.5.91.190:37401)
0620 20:08:48.637083 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:48.637087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:48.637104 (+ 17us) server_negotiation.cc:407] Connection header received
0620 20:08:48.637261 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:48.637265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:48.637320 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:48.637407 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:48.638033 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:48.638628 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:48.639969 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:48.640319 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:48.640479 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:48.641692 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:48.641695 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:48.641698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:08:48.642298 (+ 600us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:48.642334 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:08:48.642351 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:48.642435 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:48.642753 (+ 318us) server_negotiation.cc:299] Negotiation successful
0620 20:08:48.642825 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260620 20:08:48.654232 8469 raft_consensus.cc:504] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "343807d90ff6423b8dfc8ce580727149" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 37401 } }
I20260620 20:08:48.654729 8469 leader_election.cc:302] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 343807d90ff6423b8dfc8ce580727149; no voters:
I20260620 20:08:48.655108 8469 leader_election.cc:288] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:08:48.655354 8481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:48.655556 8481 raft_consensus.cc:686] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149 [term 1 LEADER]: Becoming Leader. State: Replica: 343807d90ff6423b8dfc8ce580727149, State: Running, Role: LEADER
I20260620 20:08:48.655999 8481 consensus_queue.cc:237] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "343807d90ff6423b8dfc8ce580727149" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 37401 } }
I20260620 20:08:48.656608 8469 sys_catalog.cc:564] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:08:48.658242 8483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 343807d90ff6423b8dfc8ce580727149. Latest consensus state: current_term: 1 leader_uuid: "343807d90ff6423b8dfc8ce580727149" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "343807d90ff6423b8dfc8ce580727149" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 37401 } } }
I20260620 20:08:48.658361 8483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149 [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:48.658198 8482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "343807d90ff6423b8dfc8ce580727149" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "343807d90ff6423b8dfc8ce580727149" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 37401 } } }
I20260620 20:08:48.658551 8482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149 [sys.catalog]: This master's current role is: LEADER
W20260620 20:08:48.662660 8496 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 20:08:48.663293 8496 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 20:08:48.663594 8495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:08:48.664739 8495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:08:48.672822 8495 catalog_manager.cc:1347] Generated new cluster ID: 70dc150197f24141b0a34ad4e1b647a3
I20260620 20:08:48.672976 8495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:08:48.694955 8495 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:08:48.697193 8495 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:08:48.707825 8495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 343807d90ff6423b8dfc8ce580727149: Generated new TSK 0
I20260620 20:08:48.708813 8495 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:08:48.725180 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 8401
2026-06-20T20:08:48Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1510 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:08:48 dist-test-slave-l8h4 krb5kdc[8502](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:08:48 dist-test-slave-l8h4 krb5kdc[8502](info): set up 2 sockets
Jun 20 20:08:48 dist-test-slave-l8h4 krb5kdc[8502](info): commencing operation
krb5kdc: starting...
W20260620 20:08:49.736071 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.989s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:08:49 dist-test-slave-l8h4 krb5kdc[8502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:08:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:08:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:08:49.875020 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:41729
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:35891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:41729
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:49.994931 8515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:49.995247 8515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:49.995306 8515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:50.000168 8515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:08:50.000262 8515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:50.000295 8515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:50.000317 8515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:08:50.000337 8515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:08:50.008077 8515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:35891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:41729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:41729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:50.009369 8515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:50.010483 8515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:50.019767 8523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:50.019845 8520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:50.019887 8521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:50.019989 8515 server_base.cc:1034] running on GCE node
I20260620 20:08:50.020951 8515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:50.022221 8515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:50.023379 8515 hybrid_clock.cc:648] HybridClock initialized: now 1781986130023250 us; error 155 us; skew 500 ppm
Jun 20 20:08:50 dist-test-slave-l8h4 krb5kdc[8502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986130, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:50.026504 8515 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:08:50.027449 8515 webserver.cc:460] Webserver started at http://127.5.91.190:42947/ using document root <none> and password file <none>
I20260620 20:08:50.028028 8515 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:50.028110 8515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:50.028314 8515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:50.030660 8515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "52307d8dee1d4a378c186ed345b04af1"
format_stamp: "Formatted at 2026-06-20 20:08:50 on dist-test-slave-l8h4"
server_key: "7c4076c646279b774b7b49931b5328a6"
server_key_iv: "c9154312396b10d2cb583cd4ababfd6b"
server_key_version: "encryptionkey@0"
I20260620 20:08:50.031212 8515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "52307d8dee1d4a378c186ed345b04af1"
format_stamp: "Formatted at 2026-06-20 20:08:50 on dist-test-slave-l8h4"
server_key: "7c4076c646279b774b7b49931b5328a6"
server_key_iv: "c9154312396b10d2cb583cd4ababfd6b"
server_key_version: "encryptionkey@0"
I20260620 20:08:50.035068 8515 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260620 20:08:50.037722 8530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:50.039642 8515 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:08:50.039870 8515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "52307d8dee1d4a378c186ed345b04af1"
format_stamp: "Formatted at 2026-06-20 20:08:50 on dist-test-slave-l8h4"
server_key: "7c4076c646279b774b7b49931b5328a6"
server_key_iv: "c9154312396b10d2cb583cd4ababfd6b"
server_key_version: "encryptionkey@0"
I20260620 20:08:50.040028 8515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:50.075054 8515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:50.078217 8515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:50.078411 8515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:50.086184 8515 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:41729
I20260620 20:08:50.086477 8582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:41729 every 8 connection(s)
I20260620 20:08:50.087231 8515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:08:50.090093 8583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:50.092859 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 8515
I20260620 20:08:50.092979 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:08:50.093261 5486 external_mini_cluster.cc:1428] Setting key 566a5cec6c0db15d615163b93179028c
I20260620 20:08:50.096450 8583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52307d8dee1d4a378c186ed345b04af1: Bootstrap starting.
I20260620 20:08:50.098992 8583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52307d8dee1d4a378c186ed345b04af1: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:50.099742 8583 log.cc:824] T 00000000000000000000000000000000 P 52307d8dee1d4a378c186ed345b04af1: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:50.101727 8583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52307d8dee1d4a378c186ed345b04af1: No bootstrap required, opened a new log
Jun 20 20:08:50 dist-test-slave-l8h4 krb5kdc[8502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:50.105969 8583 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52307d8dee1d4a378c186ed345b04af1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52307d8dee1d4a378c186ed345b04af1" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41729 } }
I20260620 20:08:50.106297 8583 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52307d8dee1d4a378c186ed345b04af1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:50.106375 8583 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52307d8dee1d4a378c186ed345b04af1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52307d8dee1d4a378c186ed345b04af1, State: Initialized, Role: FOLLOWER
I20260620 20:08:50.106844 8583 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52307d8dee1d4a378c186ed345b04af1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52307d8dee1d4a378c186ed345b04af1" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41729 } }
I20260620 20:08:50.106982 8583 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52307d8dee1d4a378c186ed345b04af1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:08:50.107049 8583 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52307d8dee1d4a378c186ed345b04af1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:08:50.107144 8583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52307d8dee1d4a378c186ed345b04af1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:50.108856 8583 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52307d8dee1d4a378c186ed345b04af1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52307d8dee1d4a378c186ed345b04af1" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41729 } }
I20260620 20:08:50.109222 8583 leader_election.cc:302] T 00000000000000000000000000000000 P 52307d8dee1d4a378c186ed345b04af1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52307d8dee1d4a378c186ed345b04af1; no voters:
I20260620 20:08:50.109557 8583 leader_election.cc:288] T 00000000000000000000000000000000 P 52307d8dee1d4a378c186ed345b04af1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:08:50.109642 8588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52307d8dee1d4a378c186ed345b04af1 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:50.109850 8588 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52307d8dee1d4a378c186ed345b04af1 [term 1 LEADER]: Becoming Leader. State: Replica: 52307d8dee1d4a378c186ed345b04af1, State: Running, Role: LEADER
I20260620 20:08:50.110291 8588 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52307d8dee1d4a378c186ed345b04af1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52307d8dee1d4a378c186ed345b04af1" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41729 } }
I20260620 20:08:50.110805 8586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:50.094974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57326 (local address 127.5.91.190:41729)
0620 20:08:50.096025 (+ 1051us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:50.096056 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:50.096112 (+ 56us) server_negotiation.cc:407] Connection header received
0620 20:08:50.097026 (+ 914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:50.097052 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:50.097528 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:50.097928 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:50.098891 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:50.100008 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:50.100901 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:50.101372 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:50.101534 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:50.104820 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:50.104829 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:50.104853 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:50.104902 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:50.107080 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:50.107690 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:50.107696 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:50.107701 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:50.107828 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:50.108434 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:50.108437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:50.108438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:50.108864 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:50.109089 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:50.109410 (+ 321us) server_negotiation.cc:299] Negotiation successful
0620 20:08:50.109883 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":842,"thread_start_us":264,"threads_started":1}
I20260620 20:08:50.111037 8583 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52307d8dee1d4a378c186ed345b04af1 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:08:50.112036 8589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52307d8dee1d4a378c186ed345b04af1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52307d8dee1d4a378c186ed345b04af1. Latest consensus state: current_term: 1 leader_uuid: "52307d8dee1d4a378c186ed345b04af1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52307d8dee1d4a378c186ed345b04af1" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41729 } } }
I20260620 20:08:50.112166 8589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52307d8dee1d4a378c186ed345b04af1 [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:50.112461 8590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52307d8dee1d4a378c186ed345b04af1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52307d8dee1d4a378c186ed345b04af1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52307d8dee1d4a378c186ed345b04af1" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41729 } } }
I20260620 20:08:50.112548 8590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52307d8dee1d4a378c186ed345b04af1 [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:50.113014 8597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:08:50.115792 8597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:08:50.121829 8597 catalog_manager.cc:1347] Generated new cluster ID: 0d0cdc04c23948d8b30e4a5b3ce962be
I20260620 20:08:50.121909 8597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:08:50.152441 8597 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:08:50.153486 8597 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:08:50.162886 8597 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52307d8dee1d4a378c186ed345b04af1: Generated new TSK 0
I20260620 20:08:50.163848 8597 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.91.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
I20260620 20:08:50.203087 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41729
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:35891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:50.328117 8609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:50.328418 8609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:50.328462 8609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:50.332715 8609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:50.332795 8609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:50.332857 8609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:08:50.338351 8609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:35891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:50.339479 8609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:50.340528 8609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:50.352296 8617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:50.352277 8614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:50.352409 8609 server_base.cc:1034] running on GCE node
W20260620 20:08:50.352283 8615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:50.353144 8609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:50.353853 8609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:50.355119 8609 hybrid_clock.cc:648] HybridClock initialized: now 1781986130355077 us; error 44 us; skew 500 ppm
Jun 20 20:08:50 dist-test-slave-l8h4 krb5kdc[8502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986130, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:50.358517 8609 init.cc:377] Logged in from keytab as kudu/127.5.91.129@KRBTEST.COM (short username kudu)
I20260620 20:08:50.359473 8609 webserver.cc:460] Webserver started at http://127.5.91.129:40851/ using document root <none> and password file <none>
I20260620 20:08:50.360085 8609 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:50.360157 8609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:50.360347 8609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:50.362475 8609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "3334d4d1f26d4714a24fdcb484dd63d9"
format_stamp: "Formatted at 2026-06-20 20:08:50 on dist-test-slave-l8h4"
server_key: "8bd0bf69c2d3ffe078a12e71b17a77b7"
server_key_iv: "d00cc9c525fe57ce6405b9bb35163433"
server_key_version: "encryptionkey@0"
I20260620 20:08:50.363044 8609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "3334d4d1f26d4714a24fdcb484dd63d9"
format_stamp: "Formatted at 2026-06-20 20:08:50 on dist-test-slave-l8h4"
server_key: "8bd0bf69c2d3ffe078a12e71b17a77b7"
server_key_iv: "d00cc9c525fe57ce6405b9bb35163433"
server_key_version: "encryptionkey@0"
I20260620 20:08:50.367637 8609 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260620 20:08:50.371098 8624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:50.373126 8609 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260620 20:08:50.373279 8609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "3334d4d1f26d4714a24fdcb484dd63d9"
format_stamp: "Formatted at 2026-06-20 20:08:50 on dist-test-slave-l8h4"
server_key: "8bd0bf69c2d3ffe078a12e71b17a77b7"
server_key_iv: "d00cc9c525fe57ce6405b9bb35163433"
server_key_version: "encryptionkey@0"
I20260620 20:08:50.373422 8609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:50.403059 8609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:50.406926 8609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:50.407136 8609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:50.408066 8609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:08:50.409065 8609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:50.409114 8609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:50.409168 8609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:50.409198 8609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:50.420809 8609 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:40971
I20260620 20:08:50.420969 8737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:40971 every 8 connection(s)
I20260620 20:08:50.421757 8609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:08:50.431159 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 8609
I20260620 20:08:50.431326 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:08:50.431630 5486 external_mini_cluster.cc:1428] Setting key a1fa9543e8f9d5ca528b045b9b505d9d
Jun 20 20:08:50 dist-test-slave-l8h4 krb5kdc[8502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986130, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:50.438881 8586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:50.424309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:60287 (local address 127.5.91.190:41729)
0620 20:08:50.424530 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:50.424536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:50.425812 (+ 1276us) server_negotiation.cc:407] Connection header received
0620 20:08:50.427262 (+ 1450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:50.427266 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:50.427322 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:50.427406 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:50.428689 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:50.429362 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:50.430757 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:50.431205 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:50.431318 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:50.434455 (+ 3137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:50.434459 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:50.434461 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:50.434485 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:50.435899 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:50.436677 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:50.436681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:50.436683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:50.436774 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:50.437165 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:50.437168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:50.437170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:50.437389 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:50.437522 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:50.438560 (+ 1038us) server_negotiation.cc:299] Negotiation successful
0620 20:08:50.438701 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260620 20:08:50.440115 8740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:50.424939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41729 (local address 127.5.91.129:60287)
0620 20:08:50.425618 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:50.425671 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:50.427005 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:50.427596 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:50.427603 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:50.428020 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:50.428537 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:50.428551 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:50.429533 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:50.429536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:50.430523 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:50.430533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:50.431430 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:50.431435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:50.431590 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:50.432307 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:50.432330 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:50.434237 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:50.436032 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:50.436040 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:50.436058 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:50.436417 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:50.436900 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:50.436903 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:50.436904 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:50.437062 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:50.437618 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:50.437630 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:50.438211 (+ 581us) client_negotiation.cc:769] Sending connection context
0620 20:08:50.438467 (+ 256us) client_negotiation.cc:240] Negotiation successful
0620 20:08:50.438868 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":130,"threads_started":1}
I20260620 20:08:50.441519 8738 heartbeater.cc:347] Connected to a master server at 127.5.91.190:41729
I20260620 20:08:50.441802 8738 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.5.91.130@KRBTEST.COM; defaulting to no policy
I20260620 20:08:50.442477 8738 heartbeater.cc:511] Master 127.5.91.190:41729 requested a full tablet report, sending...
I20260620 20:08:50.444545 8545 ts_manager.cc:194] Registered new tserver with Master: 3334d4d1f26d4714a24fdcb484dd63d9 (127.5.91.129:40971)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.130@KRBTEST.COM" created.
I20260620 20:08:50.446053 8545 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:60287
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
I20260620 20:08:50.465427 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41729
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:35891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:50.587446 8743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:50.588096 8743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:50.588157 8743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:50.592825 8743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:50.592904 8743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:50.592969 8743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:08:50.599076 8743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:35891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:50.600765 8743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:50.602283 8743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:50.613178 8751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:50.613178 8748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:50.613526 8749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:50.613837 8743 server_base.cc:1034] running on GCE node
I20260620 20:08:50.614313 8743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:50.614992 8743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:50.616207 8743 hybrid_clock.cc:648] HybridClock initialized: now 1781986130616187 us; error 52 us; skew 500 ppm
Jun 20 20:08:50 dist-test-slave-l8h4 krb5kdc[8502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986130, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:50.619449 8743 init.cc:377] Logged in from keytab as kudu/127.5.91.130@KRBTEST.COM (short username kudu)
I20260620 20:08:50.620409 8743 webserver.cc:460] Webserver started at http://127.5.91.130:43977/ using document root <none> and password file <none>
I20260620 20:08:50.620982 8743 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:50.621052 8743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:50.621268 8743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:50.623550 8743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "6b9273d4a9614816bdc95329cadacb96"
format_stamp: "Formatted at 2026-06-20 20:08:50 on dist-test-slave-l8h4"
server_key: "481eecc15d5af0166e562a55140d76e5"
server_key_iv: "7faf1ce83cc3230fcd1541cc342f90e9"
server_key_version: "encryptionkey@0"
I20260620 20:08:50.624122 8743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "6b9273d4a9614816bdc95329cadacb96"
format_stamp: "Formatted at 2026-06-20 20:08:50 on dist-test-slave-l8h4"
server_key: "481eecc15d5af0166e562a55140d76e5"
server_key_iv: "7faf1ce83cc3230fcd1541cc342f90e9"
server_key_version: "encryptionkey@0"
I20260620 20:08:50.628353 8743 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 20:08:50.631419 8758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:50.633376 8743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 20:08:50.633711 8743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "6b9273d4a9614816bdc95329cadacb96"
format_stamp: "Formatted at 2026-06-20 20:08:50 on dist-test-slave-l8h4"
server_key: "481eecc15d5af0166e562a55140d76e5"
server_key_iv: "7faf1ce83cc3230fcd1541cc342f90e9"
server_key_version: "encryptionkey@0"
I20260620 20:08:50.633880 8743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:50.662459 8743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:50.667309 8743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:50.667562 8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:50.668303 8743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:08:50.669565 8743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:50.669672 8743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:50.669740 8743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:50.669770 8743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:50.681035 8743 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:46673
I20260620 20:08:50.681274 8871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:46673 every 8 connection(s)
I20260620 20:08:50.681980 8743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:08:50.683130 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 8743
I20260620 20:08:50.683230 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:08:50.683504 5486 external_mini_cluster.cc:1428] Setting key 6234c6eb7770da3c447c007f3e275ccf
WARNING: no policy specified for kudu/127.5.91.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.131@KRBTEST.COM" created.
Jun 20 20:08:50 dist-test-slave-l8h4 krb5kdc[8502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986130, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:50.703964 8586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:50.684101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:56959 (local address 127.5.91.190:41729)
0620 20:08:50.684248 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:50.684253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:50.685225 (+ 972us) server_negotiation.cc:407] Connection header received
0620 20:08:50.687525 (+ 2300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:50.687627 (+ 102us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:50.687748 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:50.688037 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:50.689746 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:50.690714 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:50.692385 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:50.692803 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:50.692978 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:50.698842 (+ 5864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:50.698846 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:50.698850 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:50.698883 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:50.700460 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:50.701108 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:50.701189 (+ 81us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:50.701190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:50.701298 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:50.701974 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:50.701977 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:50.701979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:50.702356 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:50.702454 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:50.703677 (+ 1223us) server_negotiation.cc:299] Negotiation successful
0620 20:08:50.703812 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 20:08:50.705870 8874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:50.684460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41729 (local address 127.5.91.130:56959)
0620 20:08:50.685066 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:50.685120 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:50.686999 (+ 1879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:50.688449 (+ 1450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:50.688461 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:50.688946 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:50.689501 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:50.689517 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:50.690932 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:50.690937 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:50.692139 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:50.692156 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:50.693096 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:50.693101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:50.693328 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:50.694457 (+ 1129us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:50.694485 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:50.698571 (+ 4086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:50.700598 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:50.700606 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:50.700625 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:50.700988 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:50.701418 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:50.701421 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:50.701423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:50.701620 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:50.702569 (+ 949us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:50.702579 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:50.702976 (+ 397us) client_negotiation.cc:769] Sending connection context
0620 20:08:50.703642 (+ 666us) client_negotiation.cc:240] Negotiation successful
0620 20:08:50.704267 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":119,"threads_started":1}
I20260620 20:08:50.707883 8872 heartbeater.cc:347] Connected to a master server at 127.5.91.190:41729
I20260620 20:08:50.708153 8872 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:50.708819 8872 heartbeater.cc:511] Master 127.5.91.190:41729 requested a full tablet report, sending...
I20260620 20:08:50.710366 8545 ts_manager.cc:194] Registered new tserver with Master: 6b9273d4a9614816bdc95329cadacb96 (127.5.91.130:46673)
I20260620 20:08:50.711184 8545 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:56959
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
I20260620 20:08:50.721609 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41729
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:35891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:50.851709 8877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:50.852284 8877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:50.852355 8877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:50.857807 8877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:50.857898 8877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:50.857971 8877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:08:50.863794 8877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:35891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:50.864893 8877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:50.865855 8877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:50.876690 8885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:50.876689 8882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:50.876689 8883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:50.877066 8877 server_base.cc:1034] running on GCE node
I20260620 20:08:50.877650 8877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:50.878432 8877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:50.879631 8877 hybrid_clock.cc:648] HybridClock initialized: now 1781986130879610 us; error 30 us; skew 500 ppm
Jun 20 20:08:50 dist-test-slave-l8h4 krb5kdc[8502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986130, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:50.883392 8877 init.cc:377] Logged in from keytab as kudu/127.5.91.131@KRBTEST.COM (short username kudu)
I20260620 20:08:50.884809 8877 webserver.cc:460] Webserver started at http://127.5.91.131:34077/ using document root <none> and password file <none>
I20260620 20:08:50.885419 8877 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:50.885499 8877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:50.885716 8877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:50.887874 8877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "1348fd0ee7294f68aeb97f8f6325f8be"
format_stamp: "Formatted at 2026-06-20 20:08:50 on dist-test-slave-l8h4"
server_key: "2d1967851b4a9a6a64847bb778fdf170"
server_key_iv: "846f86b616dc4abcbf4601d2bcd656d1"
server_key_version: "encryptionkey@0"
I20260620 20:08:50.888427 8877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "1348fd0ee7294f68aeb97f8f6325f8be"
format_stamp: "Formatted at 2026-06-20 20:08:50 on dist-test-slave-l8h4"
server_key: "2d1967851b4a9a6a64847bb778fdf170"
server_key_iv: "846f86b616dc4abcbf4601d2bcd656d1"
server_key_version: "encryptionkey@0"
I20260620 20:08:50.892556 8877 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260620 20:08:50.895458 8892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:50.897384 8877 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 20:08:50.897579 8877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "1348fd0ee7294f68aeb97f8f6325f8be"
format_stamp: "Formatted at 2026-06-20 20:08:50 on dist-test-slave-l8h4"
server_key: "2d1967851b4a9a6a64847bb778fdf170"
server_key_iv: "846f86b616dc4abcbf4601d2bcd656d1"
server_key_version: "encryptionkey@0"
I20260620 20:08:50.897766 8877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:50.919056 8877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:50.922761 8877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:50.922956 8877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:50.923584 8877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:08:50.924449 8877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:50.924500 8877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:50.924538 8877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:50.924553 8877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:50.934675 8877 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:43861
I20260620 20:08:50.934779 9005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:43861 every 8 connection(s)
I20260620 20:08:50.936167 8877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:08:50.941020 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 8877
I20260620 20:08:50.941146 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:08:50.941416 5486 external_mini_cluster.cc:1428] Setting key 07334daf3160b0404eae519d52d7db5a
Jun 20 20:08:50 dist-test-slave-l8h4 krb5kdc[8502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986130, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:50.952536 8586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:50.938105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:57553 (local address 127.5.91.190:41729)
0620 20:08:50.938301 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:50.938306 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:50.939135 (+ 829us) server_negotiation.cc:407] Connection header received
0620 20:08:50.940769 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:50.940773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:50.940837 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:50.940986 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:50.942523 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:50.943245 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:50.944987 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:50.945233 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:50.945324 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:50.948443 (+ 3119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:50.948447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:50.948449 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:50.948472 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:50.950281 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:50.950916 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:50.950919 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:50.950920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:50.951001 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:50.951364 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:50.951393 (+ 29us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:50.951394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:50.951602 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:50.951676 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:50.952300 (+ 624us) server_negotiation.cc:299] Negotiation successful
0620 20:08:50.952407 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260620 20:08:50.953754 9008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:50.938470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41729 (local address 127.5.91.131:57553)
0620 20:08:50.938968 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:50.939021 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:50.940361 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:50.941252 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:50.941261 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:50.941666 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:50.942302 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:50.942320 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:50.943447 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:50.943452 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:50.944857 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:50.944872 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:50.945548 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:50.945560 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:50.945726 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:50.946466 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:50.946483 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:50.948255 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:50.950452 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:50.950457 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:50.950472 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:50.950787 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:50.951104 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:50.951106 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:50.951108 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:50.951257 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:50.951701 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:50.951707 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:50.952021 (+ 314us) client_negotiation.cc:769] Sending connection context
0620 20:08:50.952181 (+ 160us) client_negotiation.cc:240] Negotiation successful
0620 20:08:50.952810 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":129,"threads_started":1}
I20260620 20:08:50.955188 9006 heartbeater.cc:347] Connected to a master server at 127.5.91.190:41729
I20260620 20:08:50.955437 9006 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:50.955988 9006 heartbeater.cc:511] Master 127.5.91.190:41729 requested a full tablet report, sending...
I20260620 20:08:50.957218 8545 ts_manager.cc:194] Registered new tserver with Master: 1348fd0ee7294f68aeb97f8f6325f8be (127.5.91.131:43861)
I20260620 20:08:50.957860 8545 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:57553
I20260620 20:08:50.968302 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 20:08:50 dist-test-slave-l8h4 krb5kdc[8502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986130, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 20 20:08:50 dist-test-slave-l8h4 krb5kdc[8502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986130, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:50.996716 8586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:50.985153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57334 (local address 127.5.91.190:41729)
0620 20:08:50.985294 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:50.985297 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:50.985359 (+ 62us) server_negotiation.cc:407] Connection header received
0620 20:08:50.985604 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:50.985617 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:50.985718 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:50.985849 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:50.986941 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:50.987690 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:50.988829 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:50.989156 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:50.989284 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:50.992550 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:50.992554 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:50.992557 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:50.992591 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:50.994568 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:50.995061 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:50.995065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:50.995066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:50.995147 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:50.995455 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:50.995457 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:50.995458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:50.995788 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:50.996108 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:50.996440 (+ 332us) server_negotiation.cc:299] Negotiation successful
0620 20:08:50.996576 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260620 20:08:50.997445 8545 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:57334
I20260620 20:08:50.998489 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 8609
I20260620 20:08:51.003597 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 8743
I20260620 20:08:51.010673 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 8877
I20260620 20:08:51.019189 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 8515
2026-06-20T20:08:51Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2293 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:08:51 dist-test-slave-l8h4 krb5kdc[9018](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:08:51 dist-test-slave-l8h4 krb5kdc[9018](info): set up 2 sockets
Jun 20 20:08:51 dist-test-slave-l8h4 krb5kdc[9018](info): commencing operation
krb5kdc: starting...
W20260620 20:08:52.031708 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:08:52 dist-test-slave-l8h4 krb5kdc[9018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:08:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:08:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:08:52.171463 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:45413
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:44309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:45413
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:52.295974 9031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:52.296283 9031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:52.296433 9031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:52.302002 9031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:08:52.302116 9031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:52.302178 9031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:52.302205 9031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:08:52.302234 9031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:08:52.308291 9031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:44309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:45413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:45413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:52.309449 9031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:52.310487 9031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:52.319195 9037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:52.319217 9036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:52.319188 9039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:52.319954 9031 server_base.cc:1034] running on GCE node
I20260620 20:08:52.320914 9031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:52.322204 9031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:52.323531 9031 hybrid_clock.cc:648] HybridClock initialized: now 1781986132323499 us; error 74 us; skew 500 ppm
Jun 20 20:08:52 dist-test-slave-l8h4 krb5kdc[9018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986132, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:52.330668 9031 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:08:52.332156 9031 webserver.cc:460] Webserver started at http://127.5.91.190:44223/ using document root <none> and password file <none>
I20260620 20:08:52.333070 9031 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:52.333205 9031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:52.333448 9031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:52.335745 9031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "1b0b28f2b130440ea640eaad02538f0e"
format_stamp: "Formatted at 2026-06-20 20:08:52 on dist-test-slave-l8h4"
server_key: "f5d7be79ec16e002565cb34904cd9dc2"
server_key_iv: "c4e1d5c67748584ae76e58b1988b4a80"
server_key_version: "encryptionkey@0"
I20260620 20:08:52.336297 9031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "1b0b28f2b130440ea640eaad02538f0e"
format_stamp: "Formatted at 2026-06-20 20:08:52 on dist-test-slave-l8h4"
server_key: "f5d7be79ec16e002565cb34904cd9dc2"
server_key_iv: "c4e1d5c67748584ae76e58b1988b4a80"
server_key_version: "encryptionkey@0"
I20260620 20:08:52.340687 9031 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 20:08:52.343751 9046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:52.345173 9031 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 20:08:52.345304 9031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "1b0b28f2b130440ea640eaad02538f0e"
format_stamp: "Formatted at 2026-06-20 20:08:52 on dist-test-slave-l8h4"
server_key: "f5d7be79ec16e002565cb34904cd9dc2"
server_key_iv: "c4e1d5c67748584ae76e58b1988b4a80"
server_key_version: "encryptionkey@0"
I20260620 20:08:52.345420 9031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:52.365729 9031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:52.368891 9031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:52.369079 9031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:52.376380 9031 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:45413
I20260620 20:08:52.376609 9098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:45413 every 8 connection(s)
I20260620 20:08:52.377401 9031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:08:52.379909 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 9031
I20260620 20:08:52.380064 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:08:52.380389 5486 external_mini_cluster.cc:1428] Setting key dffd9453c63cca287c7699632ee7b7e8
I20260620 20:08:52.380470 9099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:52.387575 9099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e: Bootstrap starting.
I20260620 20:08:52.390445 9099 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e: Neither blocks nor log segments found. Creating new log.
Jun 20 20:08:52 dist-test-slave-l8h4 krb5kdc[9018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:52.391157 9099 log.cc:824] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:52.393599 9099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e: No bootstrap required, opened a new log
I20260620 20:08:52.397936 9102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:52.382241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42066 (local address 127.5.91.190:45413)
0620 20:08:52.382698 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:52.382715 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:52.382753 (+ 38us) server_negotiation.cc:407] Connection header received
0620 20:08:52.383541 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:52.383559 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:52.383892 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:52.384170 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:52.385129 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:52.386222 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:52.387396 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:52.387954 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:52.388081 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:52.391344 (+ 3263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:52.391353 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:52.391385 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:52.391437 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:52.393728 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:52.394342 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:52.394349 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:52.394357 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:52.394499 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:52.394917 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:52.394920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:52.394921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:52.395347 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:52.395512 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:52.396287 (+ 775us) server_negotiation.cc:299] Negotiation successful
0620 20:08:52.396829 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":130,"threads_started":1}
I20260620 20:08:52.398130 9099 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0b28f2b130440ea640eaad02538f0e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 45413 } }
I20260620 20:08:52.398495 9099 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:52.398557 9099 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b0b28f2b130440ea640eaad02538f0e, State: Initialized, Role: FOLLOWER
I20260620 20:08:52.399030 9099 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0b28f2b130440ea640eaad02538f0e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 45413 } }
I20260620 20:08:52.399189 9099 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:08:52.399251 9099 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:08:52.399350 9099 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:52.400610 9099 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0b28f2b130440ea640eaad02538f0e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 45413 } }
I20260620 20:08:52.400951 9099 leader_election.cc:302] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b0b28f2b130440ea640eaad02538f0e; no voters:
I20260620 20:08:52.401260 9099 leader_election.cc:288] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:08:52.401367 9104 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:52.401592 9104 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e [term 1 LEADER]: Becoming Leader. State: Replica: 1b0b28f2b130440ea640eaad02538f0e, State: Running, Role: LEADER
I20260620 20:08:52.401911 9104 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0b28f2b130440ea640eaad02538f0e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 45413 } }
I20260620 20:08:52.402470 9099 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:08:52.403409 9105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b0b28f2b130440ea640eaad02538f0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0b28f2b130440ea640eaad02538f0e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 45413 } } }
I20260620 20:08:52.403525 9105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:52.403822 9106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b0b28f2b130440ea640eaad02538f0e. Latest consensus state: current_term: 1 leader_uuid: "1b0b28f2b130440ea640eaad02538f0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0b28f2b130440ea640eaad02538f0e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 45413 } } }
I20260620 20:08:52.403919 9106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e [sys.catalog]: This master's current role is: LEADER
W20260620 20:08:52.407218 9120 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 20:08:52.407289 9120 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 20:08:52.407361 9113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:08:52.408025 9113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:08:52.413252 9113 catalog_manager.cc:1347] Generated new cluster ID: 06ac007871ee4412a25a84efa039b583
I20260620 20:08:52.413341 9113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:08:52.425546 9113 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:08:52.426672 9113 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:08:52.432005 9113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b0b28f2b130440ea640eaad02538f0e: Generated new TSK 0
I20260620 20:08:52.432693 9113 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.91.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
I20260620 20:08:52.466199 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:45413
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:44309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:52.587808 9125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:52.588101 9125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:52.588145 9125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:52.593259 9125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:52.593360 9125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:52.593443 9125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:08:52.599545 9125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:44309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:45413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:52.600661 9125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:52.601657 9125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:52.611634 9131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:52.611634 9130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:52.611651 9133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:52.612323 9125 server_base.cc:1034] running on GCE node
I20260620 20:08:52.612797 9125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:52.613441 9125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:52.614657 9125 hybrid_clock.cc:648] HybridClock initialized: now 1781986132614622 us; error 52 us; skew 500 ppm
Jun 20 20:08:52 dist-test-slave-l8h4 krb5kdc[9018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986132, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:52.617941 9125 init.cc:377] Logged in from keytab as kudu/127.5.91.129@KRBTEST.COM (short username kudu)
I20260620 20:08:52.618988 9125 webserver.cc:460] Webserver started at http://127.5.91.129:33405/ using document root <none> and password file <none>
I20260620 20:08:52.619549 9125 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:52.619621 9125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:52.619886 9125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:52.622283 9125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "bd554fb9c2634ffab401083aebab6b19"
format_stamp: "Formatted at 2026-06-20 20:08:52 on dist-test-slave-l8h4"
server_key: "5c2e8e6620ce4b334aa9b72bcda23260"
server_key_iv: "e7de8f0362071f1746b2aae5116ffa10"
server_key_version: "encryptionkey@0"
I20260620 20:08:52.622962 9125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "bd554fb9c2634ffab401083aebab6b19"
format_stamp: "Formatted at 2026-06-20 20:08:52 on dist-test-slave-l8h4"
server_key: "5c2e8e6620ce4b334aa9b72bcda23260"
server_key_iv: "e7de8f0362071f1746b2aae5116ffa10"
server_key_version: "encryptionkey@0"
I20260620 20:08:52.627341 9125 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260620 20:08:52.630375 9140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:52.631860 9125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 20:08:52.631999 9125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "bd554fb9c2634ffab401083aebab6b19"
format_stamp: "Formatted at 2026-06-20 20:08:52 on dist-test-slave-l8h4"
server_key: "5c2e8e6620ce4b334aa9b72bcda23260"
server_key_iv: "e7de8f0362071f1746b2aae5116ffa10"
server_key_version: "encryptionkey@0"
I20260620 20:08:52.632108 9125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:52.648029 9125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:52.651835 9125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:52.652051 9125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:52.652650 9125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:08:52.653488 9125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:52.653538 9125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:52.653592 9125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:52.653621 9125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:52.662673 9253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:37263 every 8 connection(s)
I20260620 20:08:52.662693 9125 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:37263
I20260620 20:08:52.663708 9125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:08:52.673522 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 9125
I20260620 20:08:52.673810 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:08:52.674357 5486 external_mini_cluster.cc:1428] Setting key 7604a44c0ae4611960839d01e788184a
Jun 20 20:08:52 dist-test-slave-l8h4 krb5kdc[9018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986132, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:52.681417 9102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:52.665661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:32837 (local address 127.5.91.190:45413)
0620 20:08:52.665857 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:52.665862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:52.666797 (+ 935us) server_negotiation.cc:407] Connection header received
0620 20:08:52.668154 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:52.668157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:52.668208 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:52.668323 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:52.669579 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:52.670200 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:52.671545 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:52.671788 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:52.671922 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:52.675974 (+ 4052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:52.675981 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:52.675984 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:52.676032 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:52.678027 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:52.679126 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:52.679130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:52.679131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:52.679256 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:52.679705 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:52.679707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:52.679709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:52.679948 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:52.680100 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:52.681061 (+ 961us) server_negotiation.cc:299] Negotiation successful
0620 20:08:52.681225 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260620 20:08:52.682446 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:52.666046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:45413 (local address 127.5.91.129:32837)
0620 20:08:52.666637 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:52.666681 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:52.667932 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:52.668454 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:52.668463 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:52.668882 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:52.669383 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:52.669397 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:52.670406 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:52.670409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:52.671335 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:52.671346 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:52.672134 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:52.672139 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:52.672470 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:52.673140 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:52.673160 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:52.675658 (+ 2498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:52.678277 (+ 2619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:52.678286 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:52.678302 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:52.678923 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:52.679370 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:52.679372 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:52.679374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:52.679588 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:52.680114 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:52.680124 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:52.680626 (+ 502us) client_negotiation.cc:769] Sending connection context
0620 20:08:52.680972 (+ 346us) client_negotiation.cc:240] Negotiation successful
0620 20:08:52.681471 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":139,"threads_started":1}
I20260620 20:08:52.683620 9254 heartbeater.cc:347] Connected to a master server at 127.5.91.190:45413
I20260620 20:08:52.683854 9254 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:52.684368 9254 heartbeater.cc:511] Master 127.5.91.190:45413 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.91.130@KRBTEST.COM; defaulting to no policy
I20260620 20:08:52.685830 9063 ts_manager.cc:194] Registered new tserver with Master: bd554fb9c2634ffab401083aebab6b19 (127.5.91.129:37263)
I20260620 20:08:52.687080 9063 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:32837
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
I20260620 20:08:52.711498 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:45413
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:44309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:52.835813 9259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:52.836093 9259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:52.836135 9259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:52.840278 9259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:52.840344 9259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:52.840407 9259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:08:52.845701 9259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:44309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:45413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:52.846753 9259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:52.848142 9259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:52.859485 9267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:52.859464 9265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:52.859432 9264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:52.859907 9259 server_base.cc:1034] running on GCE node
I20260620 20:08:52.860342 9259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:52.861058 9259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:52.862317 9259 hybrid_clock.cc:648] HybridClock initialized: now 1781986132862280 us; error 40 us; skew 500 ppm
Jun 20 20:08:52 dist-test-slave-l8h4 krb5kdc[9018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986132, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:52.866505 9259 init.cc:377] Logged in from keytab as kudu/127.5.91.130@KRBTEST.COM (short username kudu)
I20260620 20:08:52.867674 9259 webserver.cc:460] Webserver started at http://127.5.91.130:45251/ using document root <none> and password file <none>
I20260620 20:08:52.868263 9259 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:52.868350 9259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:52.868561 9259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:52.870914 9259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "7427049de8ec461ea1471916813181cc"
format_stamp: "Formatted at 2026-06-20 20:08:52 on dist-test-slave-l8h4"
server_key: "aef127d60cf6804495a4eaee44a4b5cb"
server_key_iv: "4a70ae5bfd3dc4be5ee2190e663527c5"
server_key_version: "encryptionkey@0"
I20260620 20:08:52.871455 9259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "7427049de8ec461ea1471916813181cc"
format_stamp: "Formatted at 2026-06-20 20:08:52 on dist-test-slave-l8h4"
server_key: "aef127d60cf6804495a4eaee44a4b5cb"
server_key_iv: "4a70ae5bfd3dc4be5ee2190e663527c5"
server_key_version: "encryptionkey@0"
I20260620 20:08:52.876094 9259 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260620 20:08:52.879514 9274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:52.881245 9259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 20:08:52.881403 9259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "7427049de8ec461ea1471916813181cc"
format_stamp: "Formatted at 2026-06-20 20:08:52 on dist-test-slave-l8h4"
server_key: "aef127d60cf6804495a4eaee44a4b5cb"
server_key_iv: "4a70ae5bfd3dc4be5ee2190e663527c5"
server_key_version: "encryptionkey@0"
I20260620 20:08:52.881526 9259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:52.901589 9259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:52.905855 9259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:52.906024 9259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:52.906670 9259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:08:52.907544 9259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:52.907590 9259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:52.907629 9259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:52.907642 9259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:52.916898 9259 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:38789
I20260620 20:08:52.916970 9387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:38789 every 8 connection(s)
I20260620 20:08:52.917774 9259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:08:52.920077 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 9259
I20260620 20:08:52.920269 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:08:52.920934 5486 external_mini_cluster.cc:1428] Setting key 84db0dfc26dcaa6ebf8ec0c46e8e9fe1
Jun 20 20:08:52 dist-test-slave-l8h4 krb5kdc[9018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986132, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.91.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.131@KRBTEST.COM" created.
I20260620 20:08:52.935148 9102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:52.919692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:33603 (local address 127.5.91.190:45413)
0620 20:08:52.919883 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:52.919888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:52.921080 (+ 1192us) server_negotiation.cc:407] Connection header received
0620 20:08:52.922591 (+ 1511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:52.922594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:52.922647 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:52.922721 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:52.924636 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:52.925503 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:52.927144 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:52.927492 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:52.927610 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:52.930541 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:52.930546 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:52.930549 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:52.930583 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:52.932339 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:52.933058 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:52.933064 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:52.933066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:52.933174 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:52.933663 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:52.933667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:52.933669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:52.934007 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:52.934127 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:52.934781 (+ 654us) server_negotiation.cc:299] Negotiation successful
0620 20:08:52.934932 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260620 20:08:52.936249 9390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:52.920198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:45413 (local address 127.5.91.130:33603)
0620 20:08:52.920899 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:52.920967 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:52.922371 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:52.922929 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:52.922939 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:52.923425 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:52.924260 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:52.924278 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:52.925662 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:52.925665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:52.926997 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:52.927018 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:52.927674 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:52.927677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:52.927841 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:52.928534 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:52.928552 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:52.930310 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:52.932503 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:52.932510 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:52.932525 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:52.932881 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:52.933280 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:52.933283 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:52.933284 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:52.933501 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:52.934168 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:52.934176 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:52.934496 (+ 320us) client_negotiation.cc:769] Sending connection context
0620 20:08:52.934687 (+ 191us) client_negotiation.cc:240] Negotiation successful
0620 20:08:52.935052 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":161,"threads_started":1}
I20260620 20:08:52.937669 9388 heartbeater.cc:347] Connected to a master server at 127.5.91.190:45413
I20260620 20:08:52.938071 9388 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:52.938894 9388 heartbeater.cc:511] Master 127.5.91.190:45413 requested a full tablet report, sending...
I20260620 20:08:52.940548 9063 ts_manager.cc:194] Registered new tserver with Master: 7427049de8ec461ea1471916813181cc (127.5.91.130:38789)
I20260620 20:08:52.941398 9063 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:33603
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
I20260620 20:08:52.954280 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:45413
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:44309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:53.078627 9393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:53.079058 9393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:53.079133 9393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:53.084805 9393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:53.084924 9393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:53.085018 9393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:08:53.091220 9393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:44309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:45413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:53.092327 9393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:53.093341 9393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:53.104066 9399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:53.104066 9401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:53.104157 9398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:53.104785 9393 server_base.cc:1034] running on GCE node
I20260620 20:08:53.105264 9393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:53.105985 9393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:53.107205 9393 hybrid_clock.cc:648] HybridClock initialized: now 1781986133107185 us; error 39 us; skew 500 ppm
Jun 20 20:08:53 dist-test-slave-l8h4 krb5kdc[9018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986133, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:53.111070 9393 init.cc:377] Logged in from keytab as kudu/127.5.91.131@KRBTEST.COM (short username kudu)
I20260620 20:08:53.112092 9393 webserver.cc:460] Webserver started at http://127.5.91.131:43253/ using document root <none> and password file <none>
I20260620 20:08:53.112668 9393 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:53.112754 9393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:53.112970 9393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:53.115168 9393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "38c164ef2df14703a3e61e95572ce798"
format_stamp: "Formatted at 2026-06-20 20:08:53 on dist-test-slave-l8h4"
server_key: "4a567743d01f76f5082bd347736a1087"
server_key_iv: "b386aab0d52c1d2167df0512e43d81ed"
server_key_version: "encryptionkey@0"
I20260620 20:08:53.115767 9393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "38c164ef2df14703a3e61e95572ce798"
format_stamp: "Formatted at 2026-06-20 20:08:53 on dist-test-slave-l8h4"
server_key: "4a567743d01f76f5082bd347736a1087"
server_key_iv: "b386aab0d52c1d2167df0512e43d81ed"
server_key_version: "encryptionkey@0"
I20260620 20:08:53.120148 9393 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 20:08:53.123844 9408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:53.125483 9393 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 20:08:53.125608 9393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "38c164ef2df14703a3e61e95572ce798"
format_stamp: "Formatted at 2026-06-20 20:08:53 on dist-test-slave-l8h4"
server_key: "4a567743d01f76f5082bd347736a1087"
server_key_iv: "b386aab0d52c1d2167df0512e43d81ed"
server_key_version: "encryptionkey@0"
I20260620 20:08:53.125732 9393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:53.136615 9393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:53.140579 9393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:53.140761 9393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:53.141358 9393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:08:53.142266 9393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:53.142325 9393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:53.142369 9393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:53.142398 9393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:53.151322 9393 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:35379
I20260620 20:08:53.151362 9521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:35379 every 8 connection(s)
I20260620 20:08:53.152184 9393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:08:53.152374 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 9393
I20260620 20:08:53.152508 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:08:53.152835 5486 external_mini_cluster.cc:1428] Setting key 607c5d69fa355cdf2201f96d59403aad
Jun 20 20:08:53 dist-test-slave-l8h4 krb5kdc[9018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986133, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:53.168319 9102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:53.154648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:36869 (local address 127.5.91.190:45413)
0620 20:08:53.154786 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:53.154789 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:53.156200 (+ 1411us) server_negotiation.cc:407] Connection header received
0620 20:08:53.157888 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:53.157892 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:53.157952 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:53.158031 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:53.159318 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:53.159864 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:53.160997 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:53.161208 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:53.161293 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:53.164095 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:53.164098 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:53.164100 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:53.164126 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:53.166120 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:53.166732 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:53.166735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:53.166737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:53.166829 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:53.167180 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:53.167183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:53.167184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:53.167400 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:53.167491 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:53.168021 (+ 530us) server_negotiation.cc:299] Negotiation successful
0620 20:08:53.168144 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 20:08:53.169144 9524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:53.155154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:45413 (local address 127.5.91.131:36869)
0620 20:08:53.155935 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:53.156005 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:53.157642 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:53.158267 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:53.158274 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:53.158660 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:53.159174 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:53.159188 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:53.159983 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:53.159988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:53.160880 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:53.160890 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:53.161348 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:53.161352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:53.161518 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:53.162340 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:53.162366 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:53.163924 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:53.166284 (+ 2360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:53.166290 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:53.166309 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:53.166588 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:53.166940 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:53.166942 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:53.166944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:53.167080 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:53.167499 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:53.167506 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:53.167785 (+ 279us) client_negotiation.cc:769] Sending connection context
0620 20:08:53.167968 (+ 183us) client_negotiation.cc:240] Negotiation successful
0620 20:08:53.168308 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":132,"threads_started":1}
I20260620 20:08:53.170265 9522 heartbeater.cc:347] Connected to a master server at 127.5.91.190:45413
I20260620 20:08:53.170507 9522 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:53.170991 9522 heartbeater.cc:511] Master 127.5.91.190:45413 requested a full tablet report, sending...
I20260620 20:08:53.172178 9063 ts_manager.cc:194] Registered new tserver with Master: 38c164ef2df14703a3e61e95572ce798 (127.5.91.131:35379)
I20260620 20:08:53.173050 9063 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:36869
I20260620 20:08:53.180473 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 20:08:53 dist-test-slave-l8h4 krb5kdc[9018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 20:08:53 dist-test-slave-l8h4 krb5kdc[9018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.129@KRBTEST.COM
I20260620 20:08:53.212412 9529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:53.198214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55228 (local address 127.5.91.129:37263)
0620 20:08:53.198920 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:53.198931 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:53.198976 (+ 45us) server_negotiation.cc:407] Connection header received
0620 20:08:53.199046 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:53.199055 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:53.199339 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:53.199529 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:53.200673 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:53.201691 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:53.202886 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:53.203421 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:53.203548 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:53.206823 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:53.206845 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:53.206867 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:53.206914 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:53.209635 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:53.210339 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:53.210346 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:53.210347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:53.210448 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:53.210972 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:53.210975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:53.210976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:53.211254 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:53.211436 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:53.212069 (+ 633us) server_negotiation.cc:299] Negotiation successful
0620 20:08:53.212227 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":329,"threads_started":1}
Jun 20 20:08:53 dist-test-slave-l8h4 krb5kdc[9018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:53.230093 9102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:53.216834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44838 (local address 127.5.91.190:45413)
0620 20:08:53.217010 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:53.217014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:53.217123 (+ 109us) server_negotiation.cc:407] Connection header received
0620 20:08:53.217248 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:53.217251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:53.217300 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:53.217370 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:53.218107 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:53.218719 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:53.219702 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:53.220168 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:53.220320 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:53.224771 (+ 4451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:53.224777 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:53.224780 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:53.224837 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:53.227183 (+ 2346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:53.227966 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:53.227971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:53.227973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:53.228170 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:53.228583 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:53.228587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:53.228589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:53.228964 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:53.229113 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:53.229651 (+ 538us) server_negotiation.cc:299] Negotiation successful
0620 20:08:53.229920 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260620 20:08:53.230726 9063 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44838
I20260620 20:08:53.231398 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 9125
I20260620 20:08:53.238305 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 9259
I20260620 20:08:53.244014 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 9393
I20260620 20:08:53.248708 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 9031
2026-06-20T20:08:53Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2231 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:08:53 dist-test-slave-l8h4 krb5kdc[9534](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:08:53 dist-test-slave-l8h4 krb5kdc[9534](info): set up 2 sockets
Jun 20 20:08:53 dist-test-slave-l8h4 krb5kdc[9534](info): commencing operation
krb5kdc: starting...
W20260620 20:08:54.271771 5486 mini_kdc.cc:121] Time spent starting KDC: real 1.000s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:08:54 dist-test-slave-l8h4 krb5kdc[9534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:08:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:08:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:08:54.401943 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:39885
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:42847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:39885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:54.512228 9547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:54.512506 9547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:54.512563 9547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:54.516611 9547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:08:54.516677 9547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:54.516703 9547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:54.516723 9547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:08:54.516752 9547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:08:54.522441 9547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:42847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:39885
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:39885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:54.523558 9547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:54.524573 9547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:54.532918 9553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:54.532976 9555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:54.532944 9552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:54.533218 9547 server_base.cc:1034] running on GCE node
I20260620 20:08:54.533900 9547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:54.535033 9547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:54.536213 9547 hybrid_clock.cc:648] HybridClock initialized: now 1781986134536193 us; error 43 us; skew 500 ppm
Jun 20 20:08:54 dist-test-slave-l8h4 krb5kdc[9534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986134, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:54.539103 9547 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:08:54.540215 9547 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:54.540333 9547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:54.540657 9547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:54.543280 9547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "0f3781a4fb994d08956ea11c45364290"
format_stamp: "Formatted at 2026-06-20 20:08:54 on dist-test-slave-l8h4"
server_key: "078d7f3e2ad5595848e9190311d1c2a2"
server_key_iv: "a89aca6f272325dfe015f68970ff27b6"
server_key_version: "encryptionkey@0"
I20260620 20:08:54.543910 9547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "0f3781a4fb994d08956ea11c45364290"
format_stamp: "Formatted at 2026-06-20 20:08:54 on dist-test-slave-l8h4"
server_key: "078d7f3e2ad5595848e9190311d1c2a2"
server_key_iv: "a89aca6f272325dfe015f68970ff27b6"
server_key_version: "encryptionkey@0"
I20260620 20:08:54.548882 9547 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260620 20:08:54.551951 9561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:54.553293 9547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 20:08:54.553434 9547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "0f3781a4fb994d08956ea11c45364290"
format_stamp: "Formatted at 2026-06-20 20:08:54 on dist-test-slave-l8h4"
server_key: "078d7f3e2ad5595848e9190311d1c2a2"
server_key_iv: "a89aca6f272325dfe015f68970ff27b6"
server_key_version: "encryptionkey@0"
I20260620 20:08:54.553547 9547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:54.569494 9547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:54.572926 9547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:54.573105 9547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:54.580303 9547 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:39885
I20260620 20:08:54.580556 9613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:39885 every 8 connection(s)
I20260620 20:08:54.580987 9547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:08:54.583740 9614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:54.589344 9614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290: Bootstrap starting.
I20260620 20:08:54.590268 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 9547
I20260620 20:08:54.590426 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:08:54.590945 5486 external_mini_cluster.cc:1428] Setting key 2da7551400ff737262c333293bfbe888
I20260620 20:08:54.593152 9614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:54.593894 9614 log.cc:824] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:54.596521 9614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290: No bootstrap required, opened a new log
Jun 20 20:08:54 dist-test-slave-l8h4 krb5kdc[9534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:54.600798 9614 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3781a4fb994d08956ea11c45364290" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39885 } }
I20260620 20:08:54.601090 9614 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:54.601137 9614 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f3781a4fb994d08956ea11c45364290, State: Initialized, Role: FOLLOWER
I20260620 20:08:54.601612 9614 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3781a4fb994d08956ea11c45364290" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39885 } }
I20260620 20:08:54.601744 9614 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:08:54.601794 9614 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:08:54.601905 9614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:54.602885 9614 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3781a4fb994d08956ea11c45364290" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39885 } }
I20260620 20:08:54.603223 9614 leader_election.cc:302] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f3781a4fb994d08956ea11c45364290; no voters:
I20260620 20:08:54.603569 9614 leader_election.cc:288] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:08:54.603698 9619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:54.603921 9619 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290 [term 1 LEADER]: Becoming Leader. State: Replica: 0f3781a4fb994d08956ea11c45364290, State: Running, Role: LEADER
I20260620 20:08:54.604293 9619 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3781a4fb994d08956ea11c45364290" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39885 } }
I20260620 20:08:54.604950 9614 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:08:54.606128 9620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f3781a4fb994d08956ea11c45364290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3781a4fb994d08956ea11c45364290" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39885 } } }
I20260620 20:08:54.606284 9620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290 [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:54.606321 9617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:54.592593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48366 (local address 127.5.91.190:39885)
0620 20:08:54.593034 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:54.593049 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:54.593086 (+ 37us) server_negotiation.cc:407] Connection header received
0620 20:08:54.593795 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:54.593814 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:54.594246 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:54.594572 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:54.595467 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:54.596455 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:54.597332 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:54.597810 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:54.597965 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:54.600583 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:54.600590 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:54.600609 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:54.600644 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:54.602963 (+ 2319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:54.603412 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:54.603418 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:54.603424 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:54.603569 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:54.603945 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:54.603948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:54.603950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:54.604458 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:54.604646 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:54.604957 (+ 311us) server_negotiation.cc:299] Negotiation successful
0620 20:08:54.605272 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":136,"threads_started":1}
I20260620 20:08:54.606602 9621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f3781a4fb994d08956ea11c45364290. Latest consensus state: current_term: 1 leader_uuid: "0f3781a4fb994d08956ea11c45364290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3781a4fb994d08956ea11c45364290" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39885 } } }
I20260620 20:08:54.606695 9621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290 [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:54.606969 9628 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:08:54.610802 9628 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:08:54.617695 9628 catalog_manager.cc:1347] Generated new cluster ID: 5e8965f08b2d49a782ec7165c4a7a671
I20260620 20:08:54.617779 9628 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:08:54.626562 9628 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:08:54.627828 9628 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:08:54.633711 9628 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290: Generated new TSK 0
I20260620 20:08:54.634618 9628 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.91.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
I20260620 20:08:54.670295 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:39885
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:42847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:54.808462 9640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:54.808739 9640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:54.808807 9640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:54.813318 9640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:54.813409 9640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:54.813476 9640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:08:54.820683 9640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:42847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:39885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:54.821789 9640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:54.822849 9640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:54.834728 9645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:54.834777 9640 server_base.cc:1034] running on GCE node
W20260620 20:08:54.834729 9648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:54.835178 9646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:54.835809 9640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:54.836644 9640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:54.837818 9640 hybrid_clock.cc:648] HybridClock initialized: now 1781986134837794 us; error 41 us; skew 500 ppm
Jun 20 20:08:54 dist-test-slave-l8h4 krb5kdc[9534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986134, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:54.842294 9640 init.cc:377] Logged in from keytab as kudu/127.5.91.129@KRBTEST.COM (short username kudu)
I20260620 20:08:54.843282 9640 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:54.843441 9640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:54.843698 9640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:54.846100 9640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "e602dbeeba224b4bbda7dd379473af83"
format_stamp: "Formatted at 2026-06-20 20:08:54 on dist-test-slave-l8h4"
server_key: "fec55f65060e50294972e951981ad63b"
server_key_iv: "e418ba0189c07bb5f41bb733a39f506a"
server_key_version: "encryptionkey@0"
I20260620 20:08:54.846688 9640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "e602dbeeba224b4bbda7dd379473af83"
format_stamp: "Formatted at 2026-06-20 20:08:54 on dist-test-slave-l8h4"
server_key: "fec55f65060e50294972e951981ad63b"
server_key_iv: "e418ba0189c07bb5f41bb733a39f506a"
server_key_version: "encryptionkey@0"
I20260620 20:08:54.850873 9640 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 20:08:54.853495 9654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:54.855171 9640 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:08:54.855314 9640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "e602dbeeba224b4bbda7dd379473af83"
format_stamp: "Formatted at 2026-06-20 20:08:54 on dist-test-slave-l8h4"
server_key: "fec55f65060e50294972e951981ad63b"
server_key_iv: "e418ba0189c07bb5f41bb733a39f506a"
server_key_version: "encryptionkey@0"
I20260620 20:08:54.855528 9640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:54.874030 9640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:54.877748 9640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:54.877911 9640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:54.878998 9640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:08:54.880288 9640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:54.880359 9640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:54.880411 9640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:54.880425 9640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:54.894129 9640 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:34327
I20260620 20:08:54.894327 9767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:34327 every 8 connection(s)
I20260620 20:08:54.895793 9640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:08:54.898326 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 9640
I20260620 20:08:54.898461 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:08:54.898772 5486 external_mini_cluster.cc:1428] Setting key d4ef754f2c247a036358c37bb230fc11
Jun 20 20:08:54 dist-test-slave-l8h4 krb5kdc[9534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986134, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.91.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.130@KRBTEST.COM" created.
I20260620 20:08:54.913832 9617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:54.898526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:35319 (local address 127.5.91.190:39885)
0620 20:08:54.898688 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:54.898693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:54.899923 (+ 1230us) server_negotiation.cc:407] Connection header received
0620 20:08:54.901148 (+ 1225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:54.901151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:54.901204 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:54.901286 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:54.902610 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:54.903157 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:54.905401 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:54.905651 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:54.905756 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:54.909183 (+ 3427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:54.909187 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:54.909189 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:54.909220 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:54.910856 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:54.911591 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:54.911595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:54.911597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:54.911824 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:54.912301 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:54.912304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:54.912305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:54.912551 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:54.912635 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:54.913526 (+ 891us) server_negotiation.cc:299] Negotiation successful
0620 20:08:54.913670 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 20:08:54.914690 9770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:54.898988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:39885 (local address 127.5.91.129:35319)
0620 20:08:54.899783 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:54.899837 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:54.900923 (+ 1086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:54.901491 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:54.901499 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:54.901864 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:54.902422 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:54.902437 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:54.903415 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:54.903421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:54.905218 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:54.905239 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:54.905845 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:54.905850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:54.906105 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:54.906808 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:54.906829 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:54.908941 (+ 2112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:54.911008 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:54.911015 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:54.911030 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:54.911305 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:54.911962 (+ 657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:54.911965 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:54.911967 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:54.912132 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:54.912718 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:54.912726 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:54.913202 (+ 476us) client_negotiation.cc:769] Sending connection context
0620 20:08:54.913420 (+ 218us) client_negotiation.cc:240] Negotiation successful
0620 20:08:54.913711 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":127,"threads_started":1}
I20260620 20:08:54.917937 9768 heartbeater.cc:347] Connected to a master server at 127.5.91.190:39885
I20260620 20:08:54.918251 9768 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:54.918728 9768 heartbeater.cc:511] Master 127.5.91.190:39885 requested a full tablet report, sending...
I20260620 20:08:54.921054 9578 ts_manager.cc:194] Registered new tserver with Master: e602dbeeba224b4bbda7dd379473af83 (127.5.91.129:34327)
I20260620 20:08:54.922886 9578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:35319
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
I20260620 20:08:54.941004 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:39885
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:42847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:55.074373 9773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:55.074668 9773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:55.074715 9773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:55.078787 9773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:55.078856 9773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:55.078919 9773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:08:55.085177 9773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:42847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:39885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:55.086289 9773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:55.087317 9773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:55.098129 9779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:55.098450 9778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:55.098534 9781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:55.098686 9773 server_base.cc:1034] running on GCE node
I20260620 20:08:55.099172 9773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:55.099839 9773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:55.101035 9773 hybrid_clock.cc:648] HybridClock initialized: now 1781986135101018 us; error 47 us; skew 500 ppm
Jun 20 20:08:55 dist-test-slave-l8h4 krb5kdc[9534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986135, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:55.104012 9773 init.cc:377] Logged in from keytab as kudu/127.5.91.130@KRBTEST.COM (short username kudu)
I20260620 20:08:55.104792 9773 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:55.104882 9773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:55.105114 9773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:55.107438 9773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "30528d1e33cb4a92b4f1bc676346114f"
format_stamp: "Formatted at 2026-06-20 20:08:55 on dist-test-slave-l8h4"
server_key: "f8cfe4ff6d7b81cb207febb30fa2c1da"
server_key_iv: "268356954270aad440fcaeefea13caea"
server_key_version: "encryptionkey@0"
I20260620 20:08:55.108189 9773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "30528d1e33cb4a92b4f1bc676346114f"
format_stamp: "Formatted at 2026-06-20 20:08:55 on dist-test-slave-l8h4"
server_key: "f8cfe4ff6d7b81cb207febb30fa2c1da"
server_key_iv: "268356954270aad440fcaeefea13caea"
server_key_version: "encryptionkey@0"
I20260620 20:08:55.112336 9773 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 20:08:55.115003 9787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:55.116858 9773 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 20:08:55.116997 9773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "30528d1e33cb4a92b4f1bc676346114f"
format_stamp: "Formatted at 2026-06-20 20:08:55 on dist-test-slave-l8h4"
server_key: "f8cfe4ff6d7b81cb207febb30fa2c1da"
server_key_iv: "268356954270aad440fcaeefea13caea"
server_key_version: "encryptionkey@0"
I20260620 20:08:55.117130 9773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:55.134516 9773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:55.137749 9773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:55.137960 9773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:55.138612 9773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:08:55.139477 9773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:55.139523 9773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:55.139560 9773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:55.139586 9773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:55.149621 9773 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:37927
I20260620 20:08:55.149677 9900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:37927 every 8 connection(s)
I20260620 20:08:55.150259 9773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:08:55.158934 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 9773
I20260620 20:08:55.159061 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:08:55.159723 5486 external_mini_cluster.cc:1428] Setting key d2e5ced54751abe10a55c1992588ebf0
Jun 20 20:08:55 dist-test-slave-l8h4 krb5kdc[9534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986135, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:55.167482 9617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:55.152634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:56303 (local address 127.5.91.190:39885)
0620 20:08:55.152801 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:55.152805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:55.153791 (+ 986us) server_negotiation.cc:407] Connection header received
0620 20:08:55.155246 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:55.155248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:55.155299 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:55.155378 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:55.156683 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.157420 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.158826 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.159083 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.159203 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:55.162927 (+ 3724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:55.162931 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:55.162933 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:55.162964 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:55.164521 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:55.165095 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:55.165098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:55.165101 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:55.165185 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:55.165951 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:55.165955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:55.165957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:55.166245 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:55.166368 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:55.167156 (+ 788us) server_negotiation.cc:299] Negotiation successful
0620 20:08:55.167298 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 20:08:55.168308 9903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:55.153040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:39885 (local address 127.5.91.130:56303)
0620 20:08:55.153624 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:55.153668 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:55.155010 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:55.155566 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:55.155573 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:55.156027 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:55.156511 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:55.156524 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.157564 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.157567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:55.158657 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:55.158667 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.159646 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.159656 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:55.159984 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:55.160857 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:55.160877 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:55.162582 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:55.164676 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:55.164682 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:55.164697 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:55.164979 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:55.165504 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:55.165509 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:55.165511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:55.165823 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:55.166421 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:55.166429 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:55.166884 (+ 455us) client_negotiation.cc:769] Sending connection context
0620 20:08:55.167054 (+ 170us) client_negotiation.cc:240] Negotiation successful
0620 20:08:55.167402 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":121,"threads_started":1}
WARNING: no policy specified for kudu/127.5.91.131@KRBTEST.COM; defaulting to no policy
I20260620 20:08:55.169507 9901 heartbeater.cc:347] Connected to a master server at 127.5.91.190:39885
I20260620 20:08:55.169766 9901 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:55.170248 9901 heartbeater.cc:511] Master 127.5.91.190:39885 requested a full tablet report, sending...
I20260620 20:08:55.171545 9578 ts_manager.cc:194] Registered new tserver with Master: 30528d1e33cb4a92b4f1bc676346114f (127.5.91.130:37927)
I20260620 20:08:55.172181 9578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:56303
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
I20260620 20:08:55.194401 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:39885
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:42847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:08:55.318984 9906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:55.319285 9906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:55.319351 9906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:55.324137 9906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:55.324227 9906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:55.324319 9906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:08:55.330034 9906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:42847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:39885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:55.331245 9906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:55.332247 9906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:55.342594 9912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:55.342594 9911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:55.342602 9914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:55.343281 9906 server_base.cc:1034] running on GCE node
I20260620 20:08:55.343745 9906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:55.344588 9906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:55.345813 9906 hybrid_clock.cc:648] HybridClock initialized: now 1781986135345734 us; error 56 us; skew 500 ppm
Jun 20 20:08:55 dist-test-slave-l8h4 krb5kdc[9534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986135, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:08:55.349651 9906 init.cc:377] Logged in from keytab as kudu/127.5.91.131@KRBTEST.COM (short username kudu)
I20260620 20:08:55.350554 9906 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:55.350629 9906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:55.350840 9906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:55.353214 9906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "3d4c241ef3454d62b074d6418a6f7379"
format_stamp: "Formatted at 2026-06-20 20:08:55 on dist-test-slave-l8h4"
server_key: "689483c3ca0c718815fe0cc14f8092d1"
server_key_iv: "38b1ce14bec768bfa55fc08e05ff8fe6"
server_key_version: "encryptionkey@0"
I20260620 20:08:55.354476 9906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "3d4c241ef3454d62b074d6418a6f7379"
format_stamp: "Formatted at 2026-06-20 20:08:55 on dist-test-slave-l8h4"
server_key: "689483c3ca0c718815fe0cc14f8092d1"
server_key_iv: "38b1ce14bec768bfa55fc08e05ff8fe6"
server_key_version: "encryptionkey@0"
I20260620 20:08:55.359758 9906 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260620 20:08:55.362438 9920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:55.363991 9906 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 20:08:55.364109 9906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "3d4c241ef3454d62b074d6418a6f7379"
format_stamp: "Formatted at 2026-06-20 20:08:55 on dist-test-slave-l8h4"
server_key: "689483c3ca0c718815fe0cc14f8092d1"
server_key_iv: "38b1ce14bec768bfa55fc08e05ff8fe6"
server_key_version: "encryptionkey@0"
I20260620 20:08:55.364230 9906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:55.380841 9906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:55.384764 9906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:55.384963 9906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:55.385636 9906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:08:55.386619 9906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:55.386667 9906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:55.386718 9906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:55.386749 9906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:55.396425 9906 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:39541
I20260620 20:08:55.396648 10033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:39541 every 8 connection(s)
I20260620 20:08:55.397513 9906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:08:55.401849 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 9906
I20260620 20:08:55.401984 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:08:55.402328 5486 external_mini_cluster.cc:1428] Setting key 42bea9e9e0265ba23fd426eb65aab8fb
Jun 20 20:08:55 dist-test-slave-l8h4 krb5kdc[9534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986135, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:08:55.412171 9617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:55.399768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:40255 (local address 127.5.91.190:39885)
0620 20:08:55.399924 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:55.399928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:55.401061 (+ 1133us) server_negotiation.cc:407] Connection header received
0620 20:08:55.401978 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:55.402004 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:55.402130 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:55.402244 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:55.403635 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.404196 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.405608 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.405824 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.405968 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:55.408617 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:55.408621 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:55.408623 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:55.408647 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:55.409974 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:55.410506 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:55.410510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:55.410512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:55.410591 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:55.410912 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:55.410915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:55.410917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:55.411105 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:55.411187 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:55.411711 (+ 524us) server_negotiation.cc:299] Negotiation successful
0620 20:08:55.411836 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260620 20:08:55.413290 10036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:55.400063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:39885 (local address 127.5.91.131:40255)
0620 20:08:55.400886 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:55.400926 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:55.401716 (+ 790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:55.402445 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:55.402454 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:55.402902 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:55.403498 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:55.403520 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.404344 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.404347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:55.405498 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:55.405508 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.406059 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.406063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:55.406243 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:55.406809 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:08:55.406825 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:55.408449 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:55.410102 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:55.410109 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:55.410127 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:55.410409 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:55.410672 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:08:55.410675 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:08:55.410677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:08:55.410814 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:08:55.411201 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:55.411208 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:55.411473 (+ 265us) client_negotiation.cc:769] Sending connection context
0620 20:08:55.411654 (+ 181us) client_negotiation.cc:240] Negotiation successful
0620 20:08:55.412173 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":124,"threads_started":1}
I20260620 20:08:55.414700 10034 heartbeater.cc:347] Connected to a master server at 127.5.91.190:39885
I20260620 20:08:55.414960 10034 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:55.415709 10034 heartbeater.cc:511] Master 127.5.91.190:39885 requested a full tablet report, sending...
I20260620 20:08:55.417130 9578 ts_manager.cc:194] Registered new tserver with Master: 3d4c241ef3454d62b074d6418a6f7379 (127.5.91.131:39541)
I20260620 20:08:55.417809 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 20:08:55.417794 9578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:40255
I20260620 20:08:55.427949 9617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:55.420038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48382 (local address 127.5.91.190:39885)
0620 20:08:55.420171 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:55.420175 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:55.420428 (+ 253us) server_negotiation.cc:407] Connection header received
0620 20:08:55.420574 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:55.420577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:55.420623 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:55.420687 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:55.421374 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.421872 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.422835 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.423061 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.423147 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:55.424512 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:55.424515 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:55.424517 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:08:55.424541 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:55.425970 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:55.426619 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:55.426622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:55.426624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:55.426702 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:08:55.427082 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:08:55.427085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:08:55.427086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:08:55.427350 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:55.427456 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:55.427674 (+ 218us) server_negotiation.cc:299] Negotiation successful
0620 20:08:55.427777 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 20:08:55.431794 9578 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48382:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:08:55.434852 9578 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:08:55.451087 10048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:55.444857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:34327 (local address 127.0.0.1:33284)
0620 20:08:55.445287 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:55.445299 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:55.445430 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:55.446176 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:55.446179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:55.446197 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:55.446291 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:55.446297 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.447699 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.447702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:55.449187 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:55.449202 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.450449 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.450453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:55.450572 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:55.450651 (+ 79us) client_negotiation.cc:769] Sending connection context
0620 20:08:55.450760 (+ 109us) client_negotiation.cc:240] Negotiation successful
0620 20:08:55.450956 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":25,"thread_start_us":91,"threads_started":1}
I20260620 20:08:55.451427 10050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:55.445287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33284 (local address 127.5.91.129:34327)
0620 20:08:55.445694 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:55.445700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:55.445722 (+ 22us) server_negotiation.cc:407] Connection header received
0620 20:08:55.445791 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:55.445796 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:55.445998 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:55.446191 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:55.446394 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.447566 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.449369 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.450294 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.450658 (+ 364us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:55.451011 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:55.451152 (+ 141us) server_negotiation.cc:299] Negotiation successful
0620 20:08:55.451286 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":69,"threads_started":1}
I20260620 20:08:55.451651 10046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:55.444295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:37927 (local address 127.0.0.1:34400)
0620 20:08:55.444680 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:55.444717 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:55.444896 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:55.446241 (+ 1345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:55.446244 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:55.446261 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:55.446351 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:55.446357 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.448138 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.448142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:55.449452 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:55.449461 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.450595 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.450599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:55.451331 (+ 732us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:55.451363 (+ 32us) client_negotiation.cc:769] Sending connection context
0620 20:08:55.451450 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 20:08:55.451523 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":1,"spinlock_wait_cycles":10880,"thread_start_us":112,"threads_started":1}
I20260620 20:08:55.452052 10049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:55.445040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34400 (local address 127.5.91.130:37927)
0620 20:08:55.445668 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:55.445678 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:55.445697 (+ 19us) server_negotiation.cc:407] Connection header received
0620 20:08:55.445765 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:55.445770 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:55.445937 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:55.446073 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:55.446474 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.447566 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.449658 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.450464 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.450580 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:55.450737 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:55.451809 (+ 1072us) server_negotiation.cc:299] Negotiation successful
0620 20:08:55.451911 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":120,"threads_started":1}
I20260620 20:08:55.452704 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:55.444144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:39541 (local address 127.0.0.1:55968)
0620 20:08:55.444668 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:55.444717 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:55.444866 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:55.445808 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:55.445814 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:55.445847 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:55.445989 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:55.445997 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.447408 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.447412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:55.450473 (+ 3061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:55.450482 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.452229 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.452233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:55.452409 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:55.452433 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 20:08:55.452503 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 20:08:55.452577 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":1461,"spinlock_wait_cycles":4864,"thread_start_us":95,"threads_started":1}
I20260620 20:08:55.454424 9835 tablet_service.cc:1467] Processing CreateTablet for tablet 404f6f043ff3413ea10d0fdf24939e56 (DEFAULT_TABLE table=test-table [id=50381739c29c4ec396cbdc48b298ea41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:55.454423 9700 tablet_service.cc:1467] Processing CreateTablet for tablet 404f6f043ff3413ea10d0fdf24939e56 (DEFAULT_TABLE table=test-table [id=50381739c29c4ec396cbdc48b298ea41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:55.455435 9700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 404f6f043ff3413ea10d0fdf24939e56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:55.455638 9835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 404f6f043ff3413ea10d0fdf24939e56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:55.458500 10047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:55.444344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55968 (local address 127.5.91.131:39541)
0620 20:08:55.445156 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:55.445168 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:55.445192 (+ 24us) server_negotiation.cc:407] Connection header received
0620 20:08:55.445269 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:55.445274 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:55.445495 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:55.445635 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:55.446107 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.447172 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.450633 (+ 3461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.452047 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.452154 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:55.452371 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:55.452542 (+ 171us) server_negotiation.cc:299] Negotiation successful
0620 20:08:55.452637 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":713,"thread_start_us":86,"threads_started":1}
I20260620 20:08:55.461330 9968 tablet_service.cc:1467] Processing CreateTablet for tablet 404f6f043ff3413ea10d0fdf24939e56 (DEFAULT_TABLE table=test-table [id=50381739c29c4ec396cbdc48b298ea41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:55.462311 10051 tablet_bootstrap.cc:492] T 404f6f043ff3413ea10d0fdf24939e56 P e602dbeeba224b4bbda7dd379473af83: Bootstrap starting.
I20260620 20:08:55.462378 9968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 404f6f043ff3413ea10d0fdf24939e56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:55.464653 10051 tablet_bootstrap.cc:654] T 404f6f043ff3413ea10d0fdf24939e56 P e602dbeeba224b4bbda7dd379473af83: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:55.465531 10051 log.cc:824] T 404f6f043ff3413ea10d0fdf24939e56 P e602dbeeba224b4bbda7dd379473af83: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:55.468254 10051 tablet_bootstrap.cc:492] T 404f6f043ff3413ea10d0fdf24939e56 P e602dbeeba224b4bbda7dd379473af83: No bootstrap required, opened a new log
I20260620 20:08:55.468720 10051 ts_tablet_manager.cc:1397] T 404f6f043ff3413ea10d0fdf24939e56 P e602dbeeba224b4bbda7dd379473af83: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260620 20:08:55.471495 10053 tablet_bootstrap.cc:492] T 404f6f043ff3413ea10d0fdf24939e56 P 3d4c241ef3454d62b074d6418a6f7379: Bootstrap starting.
I20260620 20:08:55.472067 10054 tablet_bootstrap.cc:492] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f: Bootstrap starting.
I20260620 20:08:55.473796 10054 tablet_bootstrap.cc:654] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:55.473862 10053 tablet_bootstrap.cc:654] T 404f6f043ff3413ea10d0fdf24939e56 P 3d4c241ef3454d62b074d6418a6f7379: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:55.473934 10051 raft_consensus.cc:348] T 404f6f043ff3413ea10d0fdf24939e56 P e602dbeeba224b4bbda7dd379473af83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d4c241ef3454d62b074d6418a6f7379" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39541 } } peers { permanent_uuid: "30528d1e33cb4a92b4f1bc676346114f" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 37927 } } peers { permanent_uuid: "e602dbeeba224b4bbda7dd379473af83" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34327 } }
I20260620 20:08:55.474254 10051 raft_consensus.cc:374] T 404f6f043ff3413ea10d0fdf24939e56 P e602dbeeba224b4bbda7dd379473af83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:55.474339 10051 raft_consensus.cc:729] T 404f6f043ff3413ea10d0fdf24939e56 P e602dbeeba224b4bbda7dd379473af83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e602dbeeba224b4bbda7dd379473af83, State: Initialized, Role: FOLLOWER
I20260620 20:08:55.474977 10054 log.cc:824] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:55.474958 10051 consensus_queue.cc:260] T 404f6f043ff3413ea10d0fdf24939e56 P e602dbeeba224b4bbda7dd379473af83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d4c241ef3454d62b074d6418a6f7379" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39541 } } peers { permanent_uuid: "30528d1e33cb4a92b4f1bc676346114f" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 37927 } } peers { permanent_uuid: "e602dbeeba224b4bbda7dd379473af83" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34327 } }
I20260620 20:08:55.475150 10053 log.cc:824] T 404f6f043ff3413ea10d0fdf24939e56 P 3d4c241ef3454d62b074d6418a6f7379: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:55.475730 9768 heartbeater.cc:503] Master 127.5.91.190:39885 was elected leader, sending a full tablet report...
I20260620 20:08:55.476984 10054 tablet_bootstrap.cc:492] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f: No bootstrap required, opened a new log
I20260620 20:08:55.476984 10053 tablet_bootstrap.cc:492] T 404f6f043ff3413ea10d0fdf24939e56 P 3d4c241ef3454d62b074d6418a6f7379: No bootstrap required, opened a new log
I20260620 20:08:55.476984 10051 ts_tablet_manager.cc:1428] T 404f6f043ff3413ea10d0fdf24939e56 P e602dbeeba224b4bbda7dd379473af83: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260620 20:08:55.477181 10053 ts_tablet_manager.cc:1397] T 404f6f043ff3413ea10d0fdf24939e56 P 3d4c241ef3454d62b074d6418a6f7379: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260620 20:08:55.477181 10054 ts_tablet_manager.cc:1397] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.005s
I20260620 20:08:55.481029 10053 raft_consensus.cc:348] T 404f6f043ff3413ea10d0fdf24939e56 P 3d4c241ef3454d62b074d6418a6f7379 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d4c241ef3454d62b074d6418a6f7379" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39541 } } peers { permanent_uuid: "30528d1e33cb4a92b4f1bc676346114f" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 37927 } } peers { permanent_uuid: "e602dbeeba224b4bbda7dd379473af83" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34327 } }
I20260620 20:08:55.481029 10054 raft_consensus.cc:348] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d4c241ef3454d62b074d6418a6f7379" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39541 } } peers { permanent_uuid: "30528d1e33cb4a92b4f1bc676346114f" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 37927 } } peers { permanent_uuid: "e602dbeeba224b4bbda7dd379473af83" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34327 } }
I20260620 20:08:55.481323 10053 raft_consensus.cc:374] T 404f6f043ff3413ea10d0fdf24939e56 P 3d4c241ef3454d62b074d6418a6f7379 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:55.481331 10054 raft_consensus.cc:374] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:55.481375 10054 raft_consensus.cc:729] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30528d1e33cb4a92b4f1bc676346114f, State: Initialized, Role: FOLLOWER
I20260620 20:08:55.481375 10053 raft_consensus.cc:729] T 404f6f043ff3413ea10d0fdf24939e56 P 3d4c241ef3454d62b074d6418a6f7379 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d4c241ef3454d62b074d6418a6f7379, State: Initialized, Role: FOLLOWER
I20260620 20:08:55.481843 10053 consensus_queue.cc:260] T 404f6f043ff3413ea10d0fdf24939e56 P 3d4c241ef3454d62b074d6418a6f7379 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d4c241ef3454d62b074d6418a6f7379" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39541 } } peers { permanent_uuid: "30528d1e33cb4a92b4f1bc676346114f" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 37927 } } peers { permanent_uuid: "e602dbeeba224b4bbda7dd379473af83" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34327 } }
I20260620 20:08:55.481843 10054 consensus_queue.cc:260] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d4c241ef3454d62b074d6418a6f7379" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39541 } } peers { permanent_uuid: "30528d1e33cb4a92b4f1bc676346114f" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 37927 } } peers { permanent_uuid: "e602dbeeba224b4bbda7dd379473af83" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34327 } }
I20260620 20:08:55.482928 10054 ts_tablet_manager.cc:1428] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260620 20:08:55.482928 10053 ts_tablet_manager.cc:1428] T 404f6f043ff3413ea10d0fdf24939e56 P 3d4c241ef3454d62b074d6418a6f7379: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260620 20:08:55.483044 10034 heartbeater.cc:503] Master 127.5.91.190:39885 was elected leader, sending a full tablet report...
I20260620 20:08:55.484041 9901 heartbeater.cc:503] Master 127.5.91.190:39885 was elected leader, sending a full tablet report...
I20260620 20:08:55.585541 10058 raft_consensus.cc:482] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:08:55.585748 10058 raft_consensus.cc:504] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d4c241ef3454d62b074d6418a6f7379" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39541 } } peers { permanent_uuid: "30528d1e33cb4a92b4f1bc676346114f" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 37927 } } peers { permanent_uuid: "e602dbeeba224b4bbda7dd379473af83" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34327 } }
I20260620 20:08:55.586846 10058 leader_election.cc:288] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d4c241ef3454d62b074d6418a6f7379 (127.5.91.131:39541), e602dbeeba224b4bbda7dd379473af83 (127.5.91.129:34327)
I20260620 20:08:55.591173 10050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:55.587429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:45011 (local address 127.5.91.129:34327)
0620 20:08:55.587543 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:55.587547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:55.587559 (+ 12us) server_negotiation.cc:407] Connection header received
0620 20:08:55.587608 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:55.587611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:55.587659 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:55.587739 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:55.588004 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.588543 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.589851 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.590518 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.590640 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:55.590693 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:55.590979 (+ 286us) server_negotiation.cc:299] Negotiation successful
0620 20:08:55.591047 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 20:08:55.591311 9903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:55.587112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:34327 (local address 127.5.91.130:45011)
0620 20:08:55.587381 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:55.587395 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:55.587485 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:55.587772 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:55.587775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:55.587788 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:55.587900 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:55.587904 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.588644 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.588647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:55.589719 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:55.589729 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.590676 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.590680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:55.590826 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:55.590860 (+ 34us) client_negotiation.cc:769] Sending connection context
0620 20:08:55.590958 (+ 98us) client_negotiation.cc:240] Negotiation successful
0620 20:08:55.591133 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202}
I20260620 20:08:55.591883 10047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:55.587191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:52713 (local address 127.5.91.131:39541)
0620 20:08:55.587311 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:55.587315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:55.587625 (+ 310us) server_negotiation.cc:407] Connection header received
0620 20:08:55.587738 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:55.587741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:55.587785 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:55.587848 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:08:55.588166 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.588915 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.590365 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.591059 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.591248 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:55.591303 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:55.591666 (+ 363us) server_negotiation.cc:299] Negotiation successful
0620 20:08:55.591762 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 20:08:55.592240 9722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "404f6f043ff3413ea10d0fdf24939e56" candidate_uuid: "30528d1e33cb4a92b4f1bc676346114f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e602dbeeba224b4bbda7dd379473af83" is_pre_election: true
I20260620 20:08:55.592383 10060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:55.587134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:39541 (local address 127.5.91.130:52713)
0620 20:08:55.587516 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:55.587528 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:55.587642 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:55.587931 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:55.587933 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:55.587948 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:08:55.588053 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:55.588059 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.589030 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.589033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:55.590051 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:55.590058 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.591388 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.591392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:55.591490 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:55.591523 (+ 33us) client_negotiation.cc:769] Sending connection context
0620 20:08:55.592039 (+ 516us) client_negotiation.cc:240] Negotiation successful
0620 20:08:55.592213 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":113,"threads_started":1}
I20260620 20:08:55.592561 9722 raft_consensus.cc:2454] T 404f6f043ff3413ea10d0fdf24939e56 P e602dbeeba224b4bbda7dd379473af83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30528d1e33cb4a92b4f1bc676346114f in term 0.
I20260620 20:08:55.593298 9790 leader_election.cc:302] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30528d1e33cb4a92b4f1bc676346114f, e602dbeeba224b4bbda7dd379473af83; no voters:
I20260620 20:08:55.593461 9988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "404f6f043ff3413ea10d0fdf24939e56" candidate_uuid: "30528d1e33cb4a92b4f1bc676346114f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d4c241ef3454d62b074d6418a6f7379" is_pre_election: true
I20260620 20:08:55.593732 9988 raft_consensus.cc:2454] T 404f6f043ff3413ea10d0fdf24939e56 P 3d4c241ef3454d62b074d6418a6f7379 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30528d1e33cb4a92b4f1bc676346114f in term 0.
I20260620 20:08:55.593847 10058 raft_consensus.cc:2781] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:08:55.593943 10058 raft_consensus.cc:482] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:08:55.593998 10058 raft_consensus.cc:3037] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:55.595094 10058 raft_consensus.cc:504] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d4c241ef3454d62b074d6418a6f7379" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39541 } } peers { permanent_uuid: "30528d1e33cb4a92b4f1bc676346114f" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 37927 } } peers { permanent_uuid: "e602dbeeba224b4bbda7dd379473af83" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34327 } }
I20260620 20:08:55.595441 10058 leader_election.cc:288] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f [CANDIDATE]: Term 1 election: Requested vote from peers 3d4c241ef3454d62b074d6418a6f7379 (127.5.91.131:39541), e602dbeeba224b4bbda7dd379473af83 (127.5.91.129:34327)
I20260620 20:08:55.595794 9988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "404f6f043ff3413ea10d0fdf24939e56" candidate_uuid: "30528d1e33cb4a92b4f1bc676346114f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d4c241ef3454d62b074d6418a6f7379"
I20260620 20:08:55.595825 9722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "404f6f043ff3413ea10d0fdf24939e56" candidate_uuid: "30528d1e33cb4a92b4f1bc676346114f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e602dbeeba224b4bbda7dd379473af83"
I20260620 20:08:55.595906 9988 raft_consensus.cc:3037] T 404f6f043ff3413ea10d0fdf24939e56 P 3d4c241ef3454d62b074d6418a6f7379 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:55.595939 9722 raft_consensus.cc:3037] T 404f6f043ff3413ea10d0fdf24939e56 P e602dbeeba224b4bbda7dd379473af83 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:55.596904 9988 raft_consensus.cc:2454] T 404f6f043ff3413ea10d0fdf24939e56 P 3d4c241ef3454d62b074d6418a6f7379 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30528d1e33cb4a92b4f1bc676346114f in term 1.
I20260620 20:08:55.597095 9722 raft_consensus.cc:2454] T 404f6f043ff3413ea10d0fdf24939e56 P e602dbeeba224b4bbda7dd379473af83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30528d1e33cb4a92b4f1bc676346114f in term 1.
I20260620 20:08:55.597203 9789 leader_election.cc:302] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30528d1e33cb4a92b4f1bc676346114f, 3d4c241ef3454d62b074d6418a6f7379; no voters:
I20260620 20:08:55.597378 10058 raft_consensus.cc:2781] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:55.597595 10058 raft_consensus.cc:686] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f [term 1 LEADER]: Becoming Leader. State: Replica: 30528d1e33cb4a92b4f1bc676346114f, State: Running, Role: LEADER
I20260620 20:08:55.597932 10058 consensus_queue.cc:237] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d4c241ef3454d62b074d6418a6f7379" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39541 } } peers { permanent_uuid: "30528d1e33cb4a92b4f1bc676346114f" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 37927 } } peers { permanent_uuid: "e602dbeeba224b4bbda7dd379473af83" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34327 } }
I20260620 20:08:55.601760 9578 catalog_manager.cc:5495] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f reported cstate change: term changed from 0 to 1, leader changed from <none> to 30528d1e33cb4a92b4f1bc676346114f (127.5.91.130). New cstate: current_term: 1 leader_uuid: "30528d1e33cb4a92b4f1bc676346114f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d4c241ef3454d62b074d6418a6f7379" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30528d1e33cb4a92b4f1bc676346114f" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e602dbeeba224b4bbda7dd379473af83" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34327 } health_report { overall_health: UNKNOWN } } }
W20260620 20:08:55.647698 9769 tablet.cc:2369] T 404f6f043ff3413ea10d0fdf24939e56 P e602dbeeba224b4bbda7dd379473af83: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 20:08:55.648648 10035 tablet.cc:2369] T 404f6f043ff3413ea10d0fdf24939e56 P 3d4c241ef3454d62b074d6418a6f7379: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:08:55.649914 10049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:55.644776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34412 (local address 127.5.91.130:37927)
0620 20:08:55.644976 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:55.644980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:55.644996 (+ 16us) server_negotiation.cc:407] Connection header received
0620 20:08:55.645271 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:55.645275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:55.645365 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:55.645491 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:08:55.645856 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.646589 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.647801 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:55.648094 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:55.648190 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:55.648749 (+ 559us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:08:55.649292 (+ 543us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:08:55.649391 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:55.649618 (+ 227us) server_negotiation.cc:299] Negotiation successful
0620 20:08:55.649748 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260620 20:08:55.651510 9902 tablet.cc:2369] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:08:55.658571 9722 raft_consensus.cc:1261] T 404f6f043ff3413ea10d0fdf24939e56 P e602dbeeba224b4bbda7dd379473af83 [term 1 FOLLOWER]: Refusing update from remote peer 30528d1e33cb4a92b4f1bc676346114f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:08:55.658566 9988 raft_consensus.cc:1261] T 404f6f043ff3413ea10d0fdf24939e56 P 3d4c241ef3454d62b074d6418a6f7379 [term 1 FOLLOWER]: Refusing update from remote peer 30528d1e33cb4a92b4f1bc676346114f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:08:55.659551 10061 consensus_queue.cc:1035] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e602dbeeba224b4bbda7dd379473af83" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:08:55.659770 10058 consensus_queue.cc:1035] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d4c241ef3454d62b074d6418a6f7379" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:08:55.670657 10065 mvcc.cc:205] Tried to move back new op lower bound from 7299015211646881792 to 7299015211411927040. Current Snapshot: MvccSnapshot[applied={T|T < 7299015211646881792}]
I20260620 20:08:55.677489 9572 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:08:55.677687 9572 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:08:55.680510 9572 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f3781a4fb994d08956ea11c45364290: Sending DeleteTablet for 3 replicas of tablet 404f6f043ff3413ea10d0fdf24939e56
I20260620 20:08:55.681653 9968 tablet_service.cc:1514] Processing DeleteTablet for tablet 404f6f043ff3413ea10d0fdf24939e56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:55 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:55968
I20260620 20:08:55.681648 9700 tablet_service.cc:1514] Processing DeleteTablet for tablet 404f6f043ff3413ea10d0fdf24939e56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:55 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:33284
I20260620 20:08:55.681916 9835 tablet_service.cc:1514] Processing DeleteTablet for tablet 404f6f043ff3413ea10d0fdf24939e56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:55 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:34400
I20260620 20:08:55.682534 10075 tablet_replica.cc:330] T 404f6f043ff3413ea10d0fdf24939e56 P 3d4c241ef3454d62b074d6418a6f7379: stopping tablet replica
I20260620 20:08:55.682680 10076 tablet_replica.cc:330] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f: stopping tablet replica
I20260620 20:08:55.682794 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 9640
I20260620 20:08:55.684062 10076 raft_consensus.cc:2229] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f [term 1 LEADER]: Raft consensus shutting down.
I20260620 20:08:55.684070 10075 raft_consensus.cc:2229] T 404f6f043ff3413ea10d0fdf24939e56 P 3d4c241ef3454d62b074d6418a6f7379 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:08:55.684495 10076 raft_consensus.cc:2258] T 404f6f043ff3413ea10d0fdf24939e56 P 30528d1e33cb4a92b4f1bc676346114f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:08:55.684526 10075 raft_consensus.cc:2258] T 404f6f043ff3413ea10d0fdf24939e56 P 3d4c241ef3454d62b074d6418a6f7379 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:08:55.685900 10075 ts_tablet_manager.cc:1905] T 404f6f043ff3413ea10d0fdf24939e56 P 3d4c241ef3454d62b074d6418a6f7379: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:08:55.688035 10075 ts_tablet_manager.cc:1918] T 404f6f043ff3413ea10d0fdf24939e56 P 3d4c241ef3454d62b074d6418a6f7379: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:08:55.688119 10075 log.cc:1196] T 404f6f043ff3413ea10d0fdf24939e56 P 3d4c241ef3454d62b074d6418a6f7379: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/404f6f043ff3413ea10d0fdf24939e56
I20260620 20:08:55.688385 10075 ts_tablet_manager.cc:1939] T 404f6f043ff3413ea10d0fdf24939e56 P 3d4c241ef3454d62b074d6418a6f7379: Deleting consensus metadata
I20260620 20:08:55.689486 9563 catalog_manager.cc:4841] TS 3d4c241ef3454d62b074d6418a6f7379 (127.5.91.131:39541): tablet 404f6f043ff3413ea10d0fdf24939e56 (table test-table [id=50381739c29c4ec396cbdc48b298ea41]) successfully deleted
W20260620 20:08:55.690519 9564 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.91.129:34327 (error 108)
I20260620 20:08:55.691509 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 9773
I20260620 20:08:55.692039 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:55.691495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:34327 (local address 127.0.0.1:33286)
0620 20:08:55.691877 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:55.691949 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.129:34327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291}
W20260620 20:08:55.693145 9564 catalog_manager.cc:4584] TS e602dbeeba224b4bbda7dd379473af83 (127.5.91.129:34327): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 404f6f043ff3413ea10d0fdf24939e56: Network error: Client connection negotiation failed: client connection to 127.5.91.129:34327: connect: Connection refused (error 111)
I20260620 20:08:55.696627 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 9906
I20260620 20:08:55.697810 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:55.697510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:37927 (local address 127.0.0.1:34416)
0620 20:08:55.697685 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:55.697733 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.130:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260620 20:08:55.702827 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 9547
2026-06-20T20:08:55Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2453 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-20T20:08:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:08:55Z Disabled control of system clock
I20260620 20:08:55.733355 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:35489
--webserver_interface=127.5.91.190
--webserver_port=0
--builtin_ntp_servers=127.5.91.148:38415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:35489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 20:08:55.851369 10081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:55.851655 10081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:55.851693 10081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:55.855741 10081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:08:55.855800 10081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:55.855813 10081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:55.855829 10081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:08:55.855873 10081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:08:55.861124 10081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:35489
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:35489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:55.862226 10081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:55.863538 10081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:55.872416 10086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:55.872416 10089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:55.872689 10081 server_base.cc:1034] running on GCE node
W20260620 20:08:55.872604 10087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:55.873313 10081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:55.874456 10081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:55.875627 10081 hybrid_clock.cc:648] HybridClock initialized: now 1781986135875602 us; error 39 us; skew 500 ppm
I20260620 20:08:55.878087 10081 webserver.cc:460] Webserver started at http://127.5.91.190:36729/ using document root <none> and password file <none>
I20260620 20:08:55.878701 10081 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:55.878772 10081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:55.879024 10081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:55.881562 10081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "e054cefa321e41d6baee4536e72daa8c"
format_stamp: "Formatted at 2026-06-20 20:08:55 on dist-test-slave-l8h4"
server_key: "3584c3ed8dc9e8159e16f8a254cbe563"
server_key_iv: "0118d30b1325c78b605bbec6fbf9e580"
server_key_version: "encryptionkey@0"
I20260620 20:08:55.882284 10081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "e054cefa321e41d6baee4536e72daa8c"
format_stamp: "Formatted at 2026-06-20 20:08:55 on dist-test-slave-l8h4"
server_key: "3584c3ed8dc9e8159e16f8a254cbe563"
server_key_iv: "0118d30b1325c78b605bbec6fbf9e580"
server_key_version: "encryptionkey@0"
I20260620 20:08:55.886976 10081 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260620 20:08:55.889827 10095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:55.891475 10081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:08:55.891595 10081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "e054cefa321e41d6baee4536e72daa8c"
format_stamp: "Formatted at 2026-06-20 20:08:55 on dist-test-slave-l8h4"
server_key: "3584c3ed8dc9e8159e16f8a254cbe563"
server_key_iv: "0118d30b1325c78b605bbec6fbf9e580"
server_key_version: "encryptionkey@0"
I20260620 20:08:55.891778 10081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:55.897289 10081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:55.898594 10081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:55.898757 10081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:55.906328 10081 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:35489
I20260620 20:08:55.906354 10147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:35489 every 8 connection(s)
I20260620 20:08:55.908471 10081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:08:55.910035 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 10081
I20260620 20:08:55.910151 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:08:55.910465 5486 external_mini_cluster.cc:1428] Setting key 1faee9c7a7e3c23fb43cd2887ee1cf49
I20260620 20:08:55.912621 10148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:55.918933 10151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:55.912762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35144 (local address 127.5.91.190:35489)
0620 20:08:55.913497 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:55.913513 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:55.913553 (+ 40us) server_negotiation.cc:407] Connection header received
0620 20:08:55.914276 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:55.914294 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:55.914660 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:55.914981 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:55.916057 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:55.916062 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:55.916075 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:08:55.916085 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:55.916143 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:08:55.916160 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:55.916277 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:55.916785 (+ 508us) server_negotiation.cc:299] Negotiation successful
0620 20:08:55.917166 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":132,"threads_started":1}
I20260620 20:08:55.919958 10148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c: Bootstrap starting.
I20260620 20:08:55.923187 10148 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:55.924033 10148 log.cc:824] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:55.927124 10148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c: No bootstrap required, opened a new log
I20260620 20:08:55.931154 10148 raft_consensus.cc:348] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e054cefa321e41d6baee4536e72daa8c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 35489 } }
I20260620 20:08:55.931449 10148 raft_consensus.cc:374] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:55.931533 10148 raft_consensus.cc:729] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e054cefa321e41d6baee4536e72daa8c, State: Initialized, Role: FOLLOWER
I20260620 20:08:55.932032 10148 consensus_queue.cc:260] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e054cefa321e41d6baee4536e72daa8c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 35489 } }
I20260620 20:08:55.932309 10148 raft_consensus.cc:388] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:08:55.932484 10148 raft_consensus.cc:482] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:08:55.932747 10148 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:55.933740 10148 raft_consensus.cc:504] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e054cefa321e41d6baee4536e72daa8c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 35489 } }
I20260620 20:08:55.934087 10148 leader_election.cc:302] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e054cefa321e41d6baee4536e72daa8c; no voters:
I20260620 20:08:55.934440 10148 leader_election.cc:288] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:08:55.934653 10153 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:55.934938 10153 raft_consensus.cc:686] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c [term 1 LEADER]: Becoming Leader. State: Replica: e054cefa321e41d6baee4536e72daa8c, State: Running, Role: LEADER
I20260620 20:08:55.935240 10153 consensus_queue.cc:237] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e054cefa321e41d6baee4536e72daa8c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 35489 } }
I20260620 20:08:55.935614 10148 sys_catalog.cc:564] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:08:55.937033 10155 sys_catalog.cc:455] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e054cefa321e41d6baee4536e72daa8c. Latest consensus state: current_term: 1 leader_uuid: "e054cefa321e41d6baee4536e72daa8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e054cefa321e41d6baee4536e72daa8c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 35489 } } }
I20260620 20:08:55.937165 10155 sys_catalog.cc:458] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:55.937888 10154 sys_catalog.cc:455] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e054cefa321e41d6baee4536e72daa8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e054cefa321e41d6baee4536e72daa8c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 35489 } } }
I20260620 20:08:55.937983 10154 sys_catalog.cc:458] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:55.938251 10164 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:08:55.940980 10164 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:08:55.947363 10164 catalog_manager.cc:1347] Generated new cluster ID: 2fa9bbb35f6544c7926068c37461cd6b
I20260620 20:08:55.947458 10164 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:08:55.976125 10164 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:08:55.976496 10164 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:08:55.982450 10164 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c: Generated new TSK 0
I20260620 20:08:55.983233 10164 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:08:55.996727 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:35489
--builtin_ntp_servers=127.5.91.148:38415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 20:08:56.119225 10172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:56.119863 10172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:56.119967 10172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:56.124262 10172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:56.124342 10172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:56.124439 10172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:08:56.129350 10172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:35489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:56.130539 10172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:56.131940 10172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:56.142561 10177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:56.142632 10172 server_base.cc:1034] running on GCE node
W20260620 20:08:56.142570 10180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:56.142833 10178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:56.143280 10172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:56.143909 10172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:56.145092 10172 hybrid_clock.cc:648] HybridClock initialized: now 1781986136145071 us; error 36 us; skew 500 ppm
I20260620 20:08:56.147127 10172 webserver.cc:460] Webserver started at http://127.5.91.129:36053/ using document root <none> and password file <none>
I20260620 20:08:56.147670 10172 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:56.147744 10172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:56.147945 10172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:56.149974 10172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "0b1b214314e348a3a22b97760a226582"
format_stamp: "Formatted at 2026-06-20 20:08:56 on dist-test-slave-l8h4"
server_key: "f6091dd6a20963b45466dfcb47fdde1f"
server_key_iv: "2be7556c99976339cd0e36d51eff9958"
server_key_version: "encryptionkey@0"
I20260620 20:08:56.150611 10172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "0b1b214314e348a3a22b97760a226582"
format_stamp: "Formatted at 2026-06-20 20:08:56 on dist-test-slave-l8h4"
server_key: "f6091dd6a20963b45466dfcb47fdde1f"
server_key_iv: "2be7556c99976339cd0e36d51eff9958"
server_key_version: "encryptionkey@0"
I20260620 20:08:56.154470 10172 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260620 20:08:56.156932 10186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:56.158579 10172 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:08:56.158684 10172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "0b1b214314e348a3a22b97760a226582"
format_stamp: "Formatted at 2026-06-20 20:08:56 on dist-test-slave-l8h4"
server_key: "f6091dd6a20963b45466dfcb47fdde1f"
server_key_iv: "2be7556c99976339cd0e36d51eff9958"
server_key_version: "encryptionkey@0"
I20260620 20:08:56.158779 10172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:56.163244 10172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:56.163859 10172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:56.164023 10172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:56.164568 10172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:08:56.165462 10172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:56.165517 10172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:56.165568 10172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:56.165596 10172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:56.175748 10172 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:35761
I20260620 20:08:56.175793 10299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:35761 every 8 connection(s)
I20260620 20:08:56.176651 10172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:08:56.183962 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 10172
I20260620 20:08:56.184134 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:08:56.184454 5486 external_mini_cluster.cc:1428] Setting key dc2337fc8823499e7e4cf5e16dd7f435
I20260620 20:08:56.184478 10151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:56.179379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:35263 (local address 127.5.91.190:35489)
0620 20:08:56.179623 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:56.179627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:56.180851 (+ 1224us) server_negotiation.cc:407] Connection header received
0620 20:08:56.182704 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:56.182707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:56.182754 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:56.182830 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:56.184004 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:56.184008 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:56.184010 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:08:56.184023 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:56.184041 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:08:56.184048 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:56.184160 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:56.184295 (+ 135us) server_negotiation.cc:299] Negotiation successful
0620 20:08:56.184339 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 20:08:56.185426 10302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:56.179749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:35489 (local address 127.5.91.129:35263)
0620 20:08:56.180647 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:56.180690 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:56.182370 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:56.183023 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:56.183030 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:56.183127 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:56.183700 (+ 573us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 20:08:56.183701 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:56.183741 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 20:08:56.183854 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:56.184143 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:56.184152 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:56.184157 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 20:08:56.184326 (+ 169us) client_negotiation.cc:240] Negotiation successful
0620 20:08:56.184573 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":265,"threads_started":1}
I20260620 20:08:56.186699 10300 heartbeater.cc:347] Connected to a master server at 127.5.91.190:35489
I20260620 20:08:56.186870 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:35489
--builtin_ntp_servers=127.5.91.148:38415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260620 20:08:56.186932 10300 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:56.187330 10300 heartbeater.cc:511] Master 127.5.91.190:35489 requested a full tablet report, sending...
I20260620 20:08:56.189352 10111 ts_manager.cc:194] Registered new tserver with Master: 0b1b214314e348a3a22b97760a226582 (127.5.91.129:35761)
W20260620 20:08:56.301537 10303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:56.301826 10303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:56.301890 10303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:56.305935 10303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:56.306003 10303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:56.306082 10303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:08:56.311033 10303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:35489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:56.312150 10303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:56.313484 10303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:56.324277 10311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:56.324369 10308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:56.324505 10303 server_base.cc:1034] running on GCE node
W20260620 20:08:56.324354 10309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:56.325350 10303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:56.326035 10303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:56.327273 10303 hybrid_clock.cc:648] HybridClock initialized: now 1781986136327242 us; error 56 us; skew 500 ppm
I20260620 20:08:56.329510 10303 webserver.cc:460] Webserver started at http://127.5.91.130:42541/ using document root <none> and password file <none>
I20260620 20:08:56.330083 10303 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:56.330188 10303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:56.330384 10303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:56.332600 10303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "ad79d987d03b4e53adc412c841d6a55d"
format_stamp: "Formatted at 2026-06-20 20:08:56 on dist-test-slave-l8h4"
server_key: "76a084b6f108a5c87835587fea7ee6a5"
server_key_iv: "f8354999905849cae4d66cc0e1fc59d9"
server_key_version: "encryptionkey@0"
I20260620 20:08:56.333482 10303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "ad79d987d03b4e53adc412c841d6a55d"
format_stamp: "Formatted at 2026-06-20 20:08:56 on dist-test-slave-l8h4"
server_key: "76a084b6f108a5c87835587fea7ee6a5"
server_key_iv: "f8354999905849cae4d66cc0e1fc59d9"
server_key_version: "encryptionkey@0"
I20260620 20:08:56.338263 10303 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260620 20:08:56.341310 10317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:56.342911 10303 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:08:56.343040 10303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "ad79d987d03b4e53adc412c841d6a55d"
format_stamp: "Formatted at 2026-06-20 20:08:56 on dist-test-slave-l8h4"
server_key: "76a084b6f108a5c87835587fea7ee6a5"
server_key_iv: "f8354999905849cae4d66cc0e1fc59d9"
server_key_version: "encryptionkey@0"
I20260620 20:08:56.343152 10303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:56.347725 10303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:56.348306 10303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:56.348474 10303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:56.349004 10303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:08:56.349872 10303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:56.349916 10303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:56.349965 10303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:56.349994 10303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:56.358354 10303 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:34993
I20260620 20:08:56.358379 10430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:34993 every 8 connection(s)
I20260620 20:08:56.359254 10303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:08:56.363168 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 10303
I20260620 20:08:56.363324 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:08:56.363639 5486 external_mini_cluster.cc:1428] Setting key 5c8aae9cdb228fe2521f7255c054cc8f
I20260620 20:08:56.365597 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:35489
--builtin_ntp_servers=127.5.91.148:38415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260620 20:08:56.365669 10151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:56.361172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:51577 (local address 127.5.91.190:35489)
0620 20:08:56.361292 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:56.361296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:56.362383 (+ 1087us) server_negotiation.cc:407] Connection header received
0620 20:08:56.363967 (+ 1584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:56.363970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:56.364035 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:56.364131 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:56.365254 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:56.365257 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:56.365260 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:08:56.365273 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:56.365294 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:08:56.365303 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:56.365377 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:56.365504 (+ 127us) server_negotiation.cc:299] Negotiation successful
0620 20:08:56.365545 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 20:08:56.366567 10433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:56.361487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:35489 (local address 127.5.91.130:51577)
0620 20:08:56.362244 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:56.362284 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:56.363725 (+ 1441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:56.364317 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:56.364326 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:56.364439 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:56.364995 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 20:08:56.364997 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:56.365049 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 20:08:56.365126 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:56.365375 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:56.365384 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:56.365389 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 20:08:56.365540 (+ 151us) client_negotiation.cc:240] Negotiation successful
0620 20:08:56.365779 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":112,"threads_started":1}
I20260620 20:08:56.367573 10431 heartbeater.cc:347] Connected to a master server at 127.5.91.190:35489
I20260620 20:08:56.367781 10431 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:56.368151 10431 heartbeater.cc:511] Master 127.5.91.190:35489 requested a full tablet report, sending...
I20260620 20:08:56.369112 10111 ts_manager.cc:194] Registered new tserver with Master: ad79d987d03b4e53adc412c841d6a55d (127.5.91.130:34993)
W20260620 20:08:56.478663 10434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:56.478961 10434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:56.479018 10434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:56.483131 10434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:56.483188 10434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:56.483264 10434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:08:56.488049 10434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:35489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:56.489107 10434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:56.490479 10434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:56.500540 10439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:56.500630 10440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:56.500761 10442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:56.501222 10434 server_base.cc:1034] running on GCE node
I20260620 20:08:56.501637 10434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:56.502327 10434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:56.503616 10434 hybrid_clock.cc:648] HybridClock initialized: now 1781986136503588 us; error 50 us; skew 500 ppm
I20260620 20:08:56.505573 10434 webserver.cc:460] Webserver started at http://127.5.91.131:40393/ using document root <none> and password file <none>
I20260620 20:08:56.506182 10434 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:56.506258 10434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:56.506453 10434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:56.508508 10434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "8e359db8f2a648f9a3657892427cc417"
format_stamp: "Formatted at 2026-06-20 20:08:56 on dist-test-slave-l8h4"
server_key: "2a5b4428055780bee3f7a197e5bf2e4c"
server_key_iv: "3c9e7b2218c386f5f7f4d7e5e403db46"
server_key_version: "encryptionkey@0"
I20260620 20:08:56.509063 10434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "8e359db8f2a648f9a3657892427cc417"
format_stamp: "Formatted at 2026-06-20 20:08:56 on dist-test-slave-l8h4"
server_key: "2a5b4428055780bee3f7a197e5bf2e4c"
server_key_iv: "3c9e7b2218c386f5f7f4d7e5e403db46"
server_key_version: "encryptionkey@0"
I20260620 20:08:56.513489 10434 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 20:08:56.516244 10448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:56.518246 10434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260620 20:08:56.518543 10434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "8e359db8f2a648f9a3657892427cc417"
format_stamp: "Formatted at 2026-06-20 20:08:56 on dist-test-slave-l8h4"
server_key: "2a5b4428055780bee3f7a197e5bf2e4c"
server_key_iv: "3c9e7b2218c386f5f7f4d7e5e403db46"
server_key_version: "encryptionkey@0"
I20260620 20:08:56.518658 10434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:56.525527 10434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:56.526715 10434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:56.526926 10434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:56.527531 10434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:08:56.528498 10434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:08:56.528546 10434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:56.528599 10434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:08:56.528626 10434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:56.539063 10434 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:45853
I20260620 20:08:56.539093 10561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:45853 every 8 connection(s)
I20260620 20:08:56.539983 10434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:08:56.541872 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 10434
I20260620 20:08:56.541991 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:08:56.542251 5486 external_mini_cluster.cc:1428] Setting key 00716e022f7daa94c9dd8bbdcf950466
I20260620 20:08:56.547005 10151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:56.541821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:40693 (local address 127.5.91.190:35489)
0620 20:08:56.542018 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:56.542023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:56.543172 (+ 1149us) server_negotiation.cc:407] Connection header received
0620 20:08:56.544967 (+ 1795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:56.544972 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:56.545113 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:56.545254 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:56.546492 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:56.546496 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:56.546498 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:08:56.546511 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:56.546538 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:08:56.546546 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:56.546637 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:56.546825 (+ 188us) server_negotiation.cc:299] Negotiation successful
0620 20:08:56.546898 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260620 20:08:56.547930 10564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:56.542303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:35489 (local address 127.5.91.131:40693)
0620 20:08:56.543029 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:56.543082 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:56.544702 (+ 1620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:56.545505 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:56.545512 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:56.545603 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:56.546220 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 20:08:56.546222 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:56.546261 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 20:08:56.546329 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:56.546649 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:56.546657 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:56.546662 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 20:08:56.546823 (+ 161us) client_negotiation.cc:240] Negotiation successful
0620 20:08:56.547047 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":110,"threads_started":1}
I20260620 20:08:56.549551 10562 heartbeater.cc:347] Connected to a master server at 127.5.91.190:35489
I20260620 20:08:56.549798 10562 heartbeater.cc:464] Registering TS with master...
I20260620 20:08:56.550237 10562 heartbeater.cc:511] Master 127.5.91.190:35489 requested a full tablet report, sending...
I20260620 20:08:56.551378 10111 ts_manager.cc:194] Registered new tserver with Master: 8e359db8f2a648f9a3657892427cc417 (127.5.91.131:45853)
I20260620 20:08:56.556761 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 20:08:56.560750 10151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:56.558947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35160 (local address 127.5.91.190:35489)
0620 20:08:56.559057 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:56.559062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:56.559338 (+ 276us) server_negotiation.cc:407] Connection header received
0620 20:08:56.559494 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:56.559497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:56.559545 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:56.559634 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:56.560291 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:56.560307 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:56.560311 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:08:56.560323 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:56.560345 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:08:56.560356 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:56.560459 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:56.560569 (+ 110us) server_negotiation.cc:299] Negotiation successful
0620 20:08:56.560614 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 20:08:56.564229 10111 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35160:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:08:56.567652 10111 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:08:56.579486 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:56.576357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:45853 (local address 127.0.0.1:41990)
0620 20:08:56.577093 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:56.577143 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:56.577238 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:56.577667 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:56.577673 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:56.577746 (+ 73us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:56.578347 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 20:08:56.578349 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:56.578371 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 20:08:56.578443 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:56.579159 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:56.579168 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:56.579174 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 20:08:56.579288 (+ 114us) client_negotiation.cc:240] Negotiation successful
0620 20:08:56.579360 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":95,"threads_started":1}
I20260620 20:08:56.579954 10575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:56.576604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41990 (local address 127.5.91.131:45853)
0620 20:08:56.577229 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:56.577240 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:56.577266 (+ 26us) server_negotiation.cc:407] Connection header received
0620 20:08:56.577320 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:56.577326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:56.577501 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:56.578035 (+ 534us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:56.578971 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:56.578977 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:56.578988 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:08:56.578997 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:56.579035 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:08:56.579057 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:56.579160 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:56.579676 (+ 516us) server_negotiation.cc:299] Negotiation successful
0620 20:08:56.579751 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":131,"threads_started":1}
I20260620 20:08:56.581087 10574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:56.577355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:34993 (local address 127.0.0.1:53734)
0620 20:08:56.579183 (+ 1828us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:56.579197 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:56.579278 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:56.579864 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:56.579867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:56.579882 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:56.580455 (+ 573us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 20:08:56.580457 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:56.580469 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 20:08:56.580495 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:56.580862 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:56.580865 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:56.580869 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 20:08:56.580935 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 20:08:56.580971 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1766,"thread_start_us":106,"threads_started":1}
I20260620 20:08:56.581329 10576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:56.577209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42916 (local address 127.5.91.129:35761)
0620 20:08:56.577642 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:56.577653 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:56.578988 (+ 1335us) server_negotiation.cc:407] Connection header received
0620 20:08:56.579048 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:56.579053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:56.579230 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:56.579363 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:56.580741 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:56.580747 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:56.580758 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:08:56.580770 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:56.580810 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:08:56.580838 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:56.580946 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:56.581099 (+ 153us) server_negotiation.cc:299] Negotiation successful
0620 20:08:56.581167 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":173,"threads_started":1}
I20260620 20:08:56.581700 10577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:56.576475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:35761 (local address 127.0.0.1:42916)
0620 20:08:56.578826 (+ 2351us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:56.578840 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:56.578969 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:56.579938 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:56.579941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:56.579957 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:56.580588 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 20:08:56.580590 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:56.580600 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 20:08:56.580625 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:56.580988 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:56.580991 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:56.580994 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 20:08:56.581555 (+ 561us) client_negotiation.cc:240] Negotiation successful
0620 20:08:56.581591 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2276,"thread_start_us":147,"threads_started":1}
I20260620 20:08:56.581804 10578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:56.578111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53734 (local address 127.5.91.130:34993)
0620 20:08:56.578551 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:56.578563 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:56.579294 (+ 731us) server_negotiation.cc:407] Connection header received
0620 20:08:56.579352 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:56.579357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:56.579515 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:56.579851 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:56.580630 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:56.580636 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:56.580647 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:08:56.580659 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:56.580697 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:08:56.580726 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:56.580823 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:56.581300 (+ 477us) server_negotiation.cc:299] Negotiation successful
0620 20:08:56.581657 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":138,"threads_started":1}
I20260620 20:08:56.583086 10496 tablet_service.cc:1467] Processing CreateTablet for tablet 138ffcc10d754fc5be2ce7cd8d3eb792 (DEFAULT_TABLE table=test-table [id=3caf3b492eb94828af29bcb11b5d555c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:56.584206 10496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 138ffcc10d754fc5be2ce7cd8d3eb792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:56.583910 10365 tablet_service.cc:1467] Processing CreateTablet for tablet 138ffcc10d754fc5be2ce7cd8d3eb792 (DEFAULT_TABLE table=test-table [id=3caf3b492eb94828af29bcb11b5d555c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:56.584195 10234 tablet_service.cc:1467] Processing CreateTablet for tablet 138ffcc10d754fc5be2ce7cd8d3eb792 (DEFAULT_TABLE table=test-table [id=3caf3b492eb94828af29bcb11b5d555c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:08:56.585302 10365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 138ffcc10d754fc5be2ce7cd8d3eb792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:56.585383 10234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 138ffcc10d754fc5be2ce7cd8d3eb792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:56.591437 10579 tablet_bootstrap.cc:492] T 138ffcc10d754fc5be2ce7cd8d3eb792 P ad79d987d03b4e53adc412c841d6a55d: Bootstrap starting.
I20260620 20:08:56.592993 10580 tablet_bootstrap.cc:492] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 8e359db8f2a648f9a3657892427cc417: Bootstrap starting.
I20260620 20:08:56.593169 10581 tablet_bootstrap.cc:492] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582: Bootstrap starting.
I20260620 20:08:56.593761 10579 tablet_bootstrap.cc:654] T 138ffcc10d754fc5be2ce7cd8d3eb792 P ad79d987d03b4e53adc412c841d6a55d: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:56.594769 10579 log.cc:824] T 138ffcc10d754fc5be2ce7cd8d3eb792 P ad79d987d03b4e53adc412c841d6a55d: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:56.595158 10580 tablet_bootstrap.cc:654] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 8e359db8f2a648f9a3657892427cc417: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:56.595158 10581 tablet_bootstrap.cc:654] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:56.595985 10581 log.cc:824] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:56.595985 10580 log.cc:824] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 8e359db8f2a648f9a3657892427cc417: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:56.597433 10579 tablet_bootstrap.cc:492] T 138ffcc10d754fc5be2ce7cd8d3eb792 P ad79d987d03b4e53adc412c841d6a55d: No bootstrap required, opened a new log
I20260620 20:08:56.597623 10579 ts_tablet_manager.cc:1397] T 138ffcc10d754fc5be2ce7cd8d3eb792 P ad79d987d03b4e53adc412c841d6a55d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 20:08:56.597880 10581 tablet_bootstrap.cc:492] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582: No bootstrap required, opened a new log
I20260620 20:08:56.598062 10581 ts_tablet_manager.cc:1397] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260620 20:08:56.598122 10580 tablet_bootstrap.cc:492] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 8e359db8f2a648f9a3657892427cc417: No bootstrap required, opened a new log
I20260620 20:08:56.598310 10580 ts_tablet_manager.cc:1397] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 8e359db8f2a648f9a3657892427cc417: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260620 20:08:56.602705 10579 raft_consensus.cc:348] T 138ffcc10d754fc5be2ce7cd8d3eb792 P ad79d987d03b4e53adc412c841d6a55d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1b214314e348a3a22b97760a226582" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 35761 } } peers { permanent_uuid: "8e359db8f2a648f9a3657892427cc417" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 45853 } } peers { permanent_uuid: "ad79d987d03b4e53adc412c841d6a55d" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 34993 } }
I20260620 20:08:56.602707 10580 raft_consensus.cc:348] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 8e359db8f2a648f9a3657892427cc417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1b214314e348a3a22b97760a226582" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 35761 } } peers { permanent_uuid: "8e359db8f2a648f9a3657892427cc417" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 45853 } } peers { permanent_uuid: "ad79d987d03b4e53adc412c841d6a55d" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 34993 } }
I20260620 20:08:56.602707 10581 raft_consensus.cc:348] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1b214314e348a3a22b97760a226582" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 35761 } } peers { permanent_uuid: "8e359db8f2a648f9a3657892427cc417" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 45853 } } peers { permanent_uuid: "ad79d987d03b4e53adc412c841d6a55d" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 34993 } }
I20260620 20:08:56.602988 10580 raft_consensus.cc:374] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 8e359db8f2a648f9a3657892427cc417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:56.602986 10581 raft_consensus.cc:374] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:56.602988 10579 raft_consensus.cc:374] T 138ffcc10d754fc5be2ce7cd8d3eb792 P ad79d987d03b4e53adc412c841d6a55d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:56.603048 10581 raft_consensus.cc:729] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1b214314e348a3a22b97760a226582, State: Initialized, Role: FOLLOWER
I20260620 20:08:56.603048 10580 raft_consensus.cc:729] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 8e359db8f2a648f9a3657892427cc417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e359db8f2a648f9a3657892427cc417, State: Initialized, Role: FOLLOWER
I20260620 20:08:56.603056 10579 raft_consensus.cc:729] T 138ffcc10d754fc5be2ce7cd8d3eb792 P ad79d987d03b4e53adc412c841d6a55d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad79d987d03b4e53adc412c841d6a55d, State: Initialized, Role: FOLLOWER
I20260620 20:08:56.603515 10579 consensus_queue.cc:260] T 138ffcc10d754fc5be2ce7cd8d3eb792 P ad79d987d03b4e53adc412c841d6a55d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1b214314e348a3a22b97760a226582" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 35761 } } peers { permanent_uuid: "8e359db8f2a648f9a3657892427cc417" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 45853 } } peers { permanent_uuid: "ad79d987d03b4e53adc412c841d6a55d" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 34993 } }
I20260620 20:08:56.603526 10581 consensus_queue.cc:260] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1b214314e348a3a22b97760a226582" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 35761 } } peers { permanent_uuid: "8e359db8f2a648f9a3657892427cc417" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 45853 } } peers { permanent_uuid: "ad79d987d03b4e53adc412c841d6a55d" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 34993 } }
I20260620 20:08:56.603526 10580 consensus_queue.cc:260] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 8e359db8f2a648f9a3657892427cc417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1b214314e348a3a22b97760a226582" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 35761 } } peers { permanent_uuid: "8e359db8f2a648f9a3657892427cc417" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 45853 } } peers { permanent_uuid: "ad79d987d03b4e53adc412c841d6a55d" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 34993 } }
I20260620 20:08:56.604238 10431 heartbeater.cc:503] Master 127.5.91.190:35489 was elected leader, sending a full tablet report...
I20260620 20:08:56.604648 10581 ts_tablet_manager.cc:1428] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260620 20:08:56.604648 10580 ts_tablet_manager.cc:1428] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 8e359db8f2a648f9a3657892427cc417: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260620 20:08:56.605266 10300 heartbeater.cc:503] Master 127.5.91.190:35489 was elected leader, sending a full tablet report...
I20260620 20:08:56.605310 10562 heartbeater.cc:503] Master 127.5.91.190:35489 was elected leader, sending a full tablet report...
I20260620 20:08:56.606789 10579 ts_tablet_manager.cc:1428] T 138ffcc10d754fc5be2ce7cd8d3eb792 P ad79d987d03b4e53adc412c841d6a55d: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
W20260620 20:08:56.610273 10432 tablet.cc:2369] T 138ffcc10d754fc5be2ce7cd8d3eb792 P ad79d987d03b4e53adc412c841d6a55d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 20:08:56.677918 10301 tablet.cc:2369] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 20:08:56.790987 10563 tablet.cc:2369] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 8e359db8f2a648f9a3657892427cc417: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:08:56.807536 10585 raft_consensus.cc:482] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:08:56.807744 10585 raft_consensus.cc:504] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1b214314e348a3a22b97760a226582" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 35761 } } peers { permanent_uuid: "8e359db8f2a648f9a3657892427cc417" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 45853 } } peers { permanent_uuid: "ad79d987d03b4e53adc412c841d6a55d" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 34993 } }
I20260620 20:08:56.808854 10585 leader_election.cc:288] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e359db8f2a648f9a3657892427cc417 (127.5.91.131:45853), ad79d987d03b4e53adc412c841d6a55d (127.5.91.130:34993)
I20260620 20:08:56.811221 10578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:56.809279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:42847 (local address 127.5.91.130:34993)
0620 20:08:56.809409 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:56.809413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:56.809849 (+ 436us) server_negotiation.cc:407] Connection header received
0620 20:08:56.809972 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:56.809976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:56.810016 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:56.810084 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:56.810848 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:56.810852 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:56.810855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:08:56.810868 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:56.810889 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:08:56.810898 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:56.810979 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:56.811044 (+ 65us) server_negotiation.cc:299] Negotiation successful
0620 20:08:56.811086 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 20:08:56.811260 10588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:56.809175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:34993 (local address 127.5.91.129:42847)
0620 20:08:56.809750 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:56.809763 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:56.809872 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:56.810122 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:56.810125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:56.810172 (+ 47us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:56.810651 (+ 479us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 20:08:56.810653 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:56.810666 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 20:08:56.810702 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:56.810972 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:56.810975 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:56.810978 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 20:08:56.811050 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 20:08:56.811086 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"mutex_wait_us":44,"thread_start_us":56,"threads_started":1}
I20260620 20:08:56.811442 10575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:56.809410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:60877 (local address 127.5.91.131:45853)
0620 20:08:56.809531 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:56.809535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:56.809938 (+ 403us) server_negotiation.cc:407] Connection header received
0620 20:08:56.810074 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:56.810077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:56.810246 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:56.810352 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:56.811059 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:56.811063 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:56.811065 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:08:56.811078 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:56.811101 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:08:56.811109 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:56.811196 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:56.811277 (+ 81us) server_negotiation.cc:299] Negotiation successful
0620 20:08:56.811317 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 20:08:56.811546 10589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:56.809175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:45853 (local address 127.5.91.129:60877)
0620 20:08:56.809757 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:56.809780 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:56.809954 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:56.810396 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:56.810400 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:56.810417 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:08:56.810828 (+ 411us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 20:08:56.810830 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:08:56.810841 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 20:08:56.810871 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:08:56.811193 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:08:56.811196 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:08:56.811199 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 20:08:56.811271 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 20:08:56.811309 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":84,"threads_started":1}
I20260620 20:08:56.812107 10385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "138ffcc10d754fc5be2ce7cd8d3eb792" candidate_uuid: "0b1b214314e348a3a22b97760a226582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad79d987d03b4e53adc412c841d6a55d" is_pre_election: true
I20260620 20:08:56.812114 10516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "138ffcc10d754fc5be2ce7cd8d3eb792" candidate_uuid: "0b1b214314e348a3a22b97760a226582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e359db8f2a648f9a3657892427cc417" is_pre_election: true
I20260620 20:08:56.812377 10516 raft_consensus.cc:2454] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 8e359db8f2a648f9a3657892427cc417 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b1b214314e348a3a22b97760a226582 in term 0.
I20260620 20:08:56.812381 10385 raft_consensus.cc:2454] T 138ffcc10d754fc5be2ce7cd8d3eb792 P ad79d987d03b4e53adc412c841d6a55d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b1b214314e348a3a22b97760a226582 in term 0.
I20260620 20:08:56.812822 10188 leader_election.cc:302] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1b214314e348a3a22b97760a226582, ad79d987d03b4e53adc412c841d6a55d; no voters:
I20260620 20:08:56.813071 10585 raft_consensus.cc:2781] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:08:56.813160 10585 raft_consensus.cc:482] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:08:56.813202 10585 raft_consensus.cc:3037] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:56.814289 10585 raft_consensus.cc:504] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1b214314e348a3a22b97760a226582" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 35761 } } peers { permanent_uuid: "8e359db8f2a648f9a3657892427cc417" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 45853 } } peers { permanent_uuid: "ad79d987d03b4e53adc412c841d6a55d" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 34993 } }
I20260620 20:08:56.814616 10585 leader_election.cc:288] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582 [CANDIDATE]: Term 1 election: Requested vote from peers 8e359db8f2a648f9a3657892427cc417 (127.5.91.131:45853), ad79d987d03b4e53adc412c841d6a55d (127.5.91.130:34993)
I20260620 20:08:56.814968 10516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "138ffcc10d754fc5be2ce7cd8d3eb792" candidate_uuid: "0b1b214314e348a3a22b97760a226582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e359db8f2a648f9a3657892427cc417"
I20260620 20:08:56.814968 10385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "138ffcc10d754fc5be2ce7cd8d3eb792" candidate_uuid: "0b1b214314e348a3a22b97760a226582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad79d987d03b4e53adc412c841d6a55d"
I20260620 20:08:56.815081 10516 raft_consensus.cc:3037] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 8e359db8f2a648f9a3657892427cc417 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:56.815102 10385 raft_consensus.cc:3037] T 138ffcc10d754fc5be2ce7cd8d3eb792 P ad79d987d03b4e53adc412c841d6a55d [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:56.816121 10385 raft_consensus.cc:2454] T 138ffcc10d754fc5be2ce7cd8d3eb792 P ad79d987d03b4e53adc412c841d6a55d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b1b214314e348a3a22b97760a226582 in term 1.
I20260620 20:08:56.816406 10188 leader_election.cc:302] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1b214314e348a3a22b97760a226582, ad79d987d03b4e53adc412c841d6a55d; no voters:
I20260620 20:08:56.816578 10585 raft_consensus.cc:2781] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:56.816816 10516 raft_consensus.cc:2454] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 8e359db8f2a648f9a3657892427cc417 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b1b214314e348a3a22b97760a226582 in term 1.
I20260620 20:08:56.816816 10585 raft_consensus.cc:686] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582 [term 1 LEADER]: Becoming Leader. State: Replica: 0b1b214314e348a3a22b97760a226582, State: Running, Role: LEADER
I20260620 20:08:56.817260 10585 consensus_queue.cc:237] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1b214314e348a3a22b97760a226582" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 35761 } } peers { permanent_uuid: "8e359db8f2a648f9a3657892427cc417" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 45853 } } peers { permanent_uuid: "ad79d987d03b4e53adc412c841d6a55d" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 34993 } }
I20260620 20:08:56.821471 10103 catalog_manager.cc:5495] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b1b214314e348a3a22b97760a226582 (127.5.91.129). New cstate: current_term: 1 leader_uuid: "0b1b214314e348a3a22b97760a226582" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1b214314e348a3a22b97760a226582" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 35761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e359db8f2a648f9a3657892427cc417" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 45853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad79d987d03b4e53adc412c841d6a55d" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 34993 } health_report { overall_health: UNKNOWN } } }
I20260620 20:08:56.881961 10576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:56.880164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42918 (local address 127.5.91.129:35761)
0620 20:08:56.880308 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:56.880312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:56.880329 (+ 17us) server_negotiation.cc:407] Connection header received
0620 20:08:56.880465 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:56.880468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:56.880637 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:56.880726 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:56.881504 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:56.881507 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:56.881509 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:08:56.881520 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:56.881538 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:08:56.881546 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:56.881635 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:56.881798 (+ 163us) server_negotiation.cc:299] Negotiation successful
0620 20:08:56.881846 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 20:08:56.888872 10516 raft_consensus.cc:1261] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 8e359db8f2a648f9a3657892427cc417 [term 1 FOLLOWER]: Refusing update from remote peer 0b1b214314e348a3a22b97760a226582: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:08:56.888871 10385 raft_consensus.cc:1261] T 138ffcc10d754fc5be2ce7cd8d3eb792 P ad79d987d03b4e53adc412c841d6a55d [term 1 FOLLOWER]: Refusing update from remote peer 0b1b214314e348a3a22b97760a226582: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:08:56.889573 10590 consensus_queue.cc:1035] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad79d987d03b4e53adc412c841d6a55d" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 34993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:08:56.889752 10585 consensus_queue.cc:1035] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 0b1b214314e348a3a22b97760a226582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e359db8f2a648f9a3657892427cc417" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 45853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:08:56.898257 10594 mvcc.cc:205] Tried to move back new op lower bound from 7299015216688865280 to 7299015216406765568. Current Snapshot: MvccSnapshot[applied={T|T < 7299015216688865280}]
I20260620 20:08:56.907358 10111 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:08:56.907549 10111 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:08:56.910257 10111 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e054cefa321e41d6baee4536e72daa8c: Sending DeleteTablet for 3 replicas of tablet 138ffcc10d754fc5be2ce7cd8d3eb792
I20260620 20:08:56.911386 10496 tablet_service.cc:1514] Processing DeleteTablet for tablet 138ffcc10d754fc5be2ce7cd8d3eb792 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:56 UTC) from {username='slave'} at 127.0.0.1:41990
I20260620 20:08:56.911455 10365 tablet_service.cc:1514] Processing DeleteTablet for tablet 138ffcc10d754fc5be2ce7cd8d3eb792 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:56 UTC) from {username='slave'} at 127.0.0.1:53734
I20260620 20:08:56.911909 10234 tablet_service.cc:1514] Processing DeleteTablet for tablet 138ffcc10d754fc5be2ce7cd8d3eb792 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:08:56 UTC) from {username='slave'} at 127.0.0.1:42916
I20260620 20:08:56.912658 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 10172
I20260620 20:08:56.913748 10603 tablet_replica.cc:330] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 8e359db8f2a648f9a3657892427cc417: stopping tablet replica
I20260620 20:08:56.914098 10603 raft_consensus.cc:2229] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 8e359db8f2a648f9a3657892427cc417 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:08:56.914549 10603 raft_consensus.cc:2258] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 8e359db8f2a648f9a3657892427cc417 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:08:56.916750 10604 tablet_replica.cc:330] T 138ffcc10d754fc5be2ce7cd8d3eb792 P ad79d987d03b4e53adc412c841d6a55d: stopping tablet replica
I20260620 20:08:56.917079 10604 raft_consensus.cc:2229] T 138ffcc10d754fc5be2ce7cd8d3eb792 P ad79d987d03b4e53adc412c841d6a55d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:08:56.917416 10604 raft_consensus.cc:2258] T 138ffcc10d754fc5be2ce7cd8d3eb792 P ad79d987d03b4e53adc412c841d6a55d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:08:56.919171 10603 ts_tablet_manager.cc:1905] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 8e359db8f2a648f9a3657892427cc417: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 20:08:56.922557 10099 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.91.129:35761 (error 108)
I20260620 20:08:56.922752 10603 ts_tablet_manager.cc:1918] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 8e359db8f2a648f9a3657892427cc417: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:08:56.922863 10603 log.cc:1196] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 8e359db8f2a648f9a3657892427cc417: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/138ffcc10d754fc5be2ce7cd8d3eb792
I20260620 20:08:56.923151 10603 ts_tablet_manager.cc:1939] T 138ffcc10d754fc5be2ce7cd8d3eb792 P 8e359db8f2a648f9a3657892427cc417: Deleting consensus metadata
I20260620 20:08:56.923436 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 10303
I20260620 20:08:56.924643 10577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:56.923812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:35761 (local address 127.0.0.1:42934)
0620 20:08:56.924436 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:56.924545 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.129:35761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445}
W20260620 20:08:56.924808 10099 catalog_manager.cc:4584] TS 0b1b214314e348a3a22b97760a226582 (127.5.91.129:35761): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 138ffcc10d754fc5be2ce7cd8d3eb792: Network error: Client connection negotiation failed: client connection to 127.5.91.129:35761: connect: Connection refused (error 111)
I20260620 20:08:56.925419 10096 catalog_manager.cc:4841] TS 8e359db8f2a648f9a3657892427cc417 (127.5.91.131:45853): tablet 138ffcc10d754fc5be2ce7cd8d3eb792 (table test-table [id=3caf3b492eb94828af29bcb11b5d555c]) successfully deleted
I20260620 20:08:56.930624 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 10434
I20260620 20:08:56.932353 10577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:56.932016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:34993 (local address 127.0.0.1:53748)
0620 20:08:56.932204 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:56.932258 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.130:34993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260620 20:08:56.935710 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 10081
2026-06-20T20:08:56Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1234 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260620 20:08:56.960286 5486 mini_oidc.cc:152] Starting JWKS server
I20260620 20:08:56.962353 5486 webserver.cc:460] Webserver started at https://127.0.0.1:36137/ using document root <none> and password file <none>
I20260620 20:08:56.962536 5486 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 20:08:56.962801 5486 webserver.cc:460] Webserver started at http://127.0.0.1:37381/ using document root <none> and password file <none>
2026-06-20T20:08:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:08:56Z Disabled control of system clock
I20260620 20:08:56.974220 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:42333
--webserver_interface=127.5.91.190
--webserver_port=0
--builtin_ntp_servers=127.5.91.148:44613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:42333
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36137/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781986111135528-5486-0/testchainca.cert with env {}
W20260620 20:08:57.102365 10612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:57.102658 10612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:57.102701 10612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:57.107344 10612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:08:57.107411 10612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:57.107426 10612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:57.107442 10612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:08:57.107458 10612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 20:08:57.107472 10612 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 20:08:57.107484 10612 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36137/jwks.json
I20260620 20:08:57.114060 10612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:44613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:42333
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:42333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36137/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781986111135528-5486-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:57.115145 10612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:57.116485 10612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:57.126688 10618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:57.126610 10620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:57.126587 10617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:57.126791 10612 server_base.cc:1034] running on GCE node
I20260620 20:08:57.127406 10612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:57.128690 10612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:57.129948 10612 hybrid_clock.cc:648] HybridClock initialized: now 1781986137129928 us; error 63 us; skew 500 ppm
I20260620 20:08:57.132037 10612 webserver.cc:460] Webserver started at http://127.5.91.190:46381/ using document root <none> and password file <none>
I20260620 20:08:57.132608 10612 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:57.132674 10612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:57.132862 10612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:57.134924 10612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "f9804bedafae47928d8652278eefb945"
format_stamp: "Formatted at 2026-06-20 20:08:57 on dist-test-slave-l8h4"
server_key: "a4e6581ed2fcc0efaeee2439cb1530a8"
server_key_iv: "3b1ae74b2f3ed9b760c2425750c734e0"
server_key_version: "encryptionkey@0"
I20260620 20:08:57.135440 10612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "f9804bedafae47928d8652278eefb945"
format_stamp: "Formatted at 2026-06-20 20:08:57 on dist-test-slave-l8h4"
server_key: "a4e6581ed2fcc0efaeee2439cb1530a8"
server_key_iv: "3b1ae74b2f3ed9b760c2425750c734e0"
server_key_version: "encryptionkey@0"
I20260620 20:08:57.139068 10612 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 20:08:57.141618 10626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:57.143151 10612 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 20:08:57.143293 10612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "f9804bedafae47928d8652278eefb945"
format_stamp: "Formatted at 2026-06-20 20:08:57 on dist-test-slave-l8h4"
server_key: "a4e6581ed2fcc0efaeee2439cb1530a8"
server_key_iv: "3b1ae74b2f3ed9b760c2425750c734e0"
server_key_version: "encryptionkey@0"
I20260620 20:08:57.143393 10612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:57.157867 10612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:57.159278 10612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:57.159711 10612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:57.167351 10612 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:42333
I20260620 20:08:57.167388 10678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:42333 every 8 connection(s)
I20260620 20:08:57.168300 10612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:08:57.171545 10679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:57.171891 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 10612
I20260620 20:08:57.172030 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:08:57.172348 5486 external_mini_cluster.cc:1428] Setting key 8ecc7234f8d6eac584c40e13e13f1a82
I20260620 20:08:57.178768 10679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9804bedafae47928d8652278eefb945: Bootstrap starting.
I20260620 20:08:57.181540 10679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9804bedafae47928d8652278eefb945: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:57.182474 10679 log.cc:824] T 00000000000000000000000000000000 P f9804bedafae47928d8652278eefb945: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:57.182722 10682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:57.173983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45318 (local address 127.5.91.190:42333)
0620 20:08:57.174580 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:57.174595 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:57.174628 (+ 33us) server_negotiation.cc:407] Connection header received
0620 20:08:57.175301 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:57.175318 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:57.175717 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:57.175998 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:57.176694 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:57.177756 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:57.178964 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:57.179682 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:57.179805 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:57.180854 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:57.180859 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:57.180871 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:08:57.181267 (+ 396us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:57.181325 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:08:57.181346 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:57.181433 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:57.181658 (+ 225us) server_negotiation.cc:299] Negotiation successful
0620 20:08:57.181918 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":130,"threads_started":1}
I20260620 20:08:57.184551 10679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9804bedafae47928d8652278eefb945: No bootstrap required, opened a new log
I20260620 20:08:57.186280 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:08:57.189023 10679 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9804bedafae47928d8652278eefb945 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9804bedafae47928d8652278eefb945" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42333 } }
I20260620 20:08:57.189294 10679 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9804bedafae47928d8652278eefb945 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:57.189344 10679 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9804bedafae47928d8652278eefb945 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9804bedafae47928d8652278eefb945, State: Initialized, Role: FOLLOWER
I20260620 20:08:57.189760 10679 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9804bedafae47928d8652278eefb945 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9804bedafae47928d8652278eefb945" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42333 } }
I20260620 20:08:57.189905 10679 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9804bedafae47928d8652278eefb945 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:08:57.189968 10679 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9804bedafae47928d8652278eefb945 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:08:57.190068 10679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9804bedafae47928d8652278eefb945 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:57.191056 10679 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9804bedafae47928d8652278eefb945 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9804bedafae47928d8652278eefb945" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42333 } }
I20260620 20:08:57.191382 10679 leader_election.cc:302] T 00000000000000000000000000000000 P f9804bedafae47928d8652278eefb945 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9804bedafae47928d8652278eefb945; no voters:
I20260620 20:08:57.191689 10679 leader_election.cc:288] T 00000000000000000000000000000000 P f9804bedafae47928d8652278eefb945 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:08:57.191963 10684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9804bedafae47928d8652278eefb945 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:57.192154 10684 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9804bedafae47928d8652278eefb945 [term 1 LEADER]: Becoming Leader. State: Replica: f9804bedafae47928d8652278eefb945, State: Running, Role: LEADER
I20260620 20:08:57.192461 10684 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9804bedafae47928d8652278eefb945 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9804bedafae47928d8652278eefb945" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42333 } }
I20260620 20:08:57.192866 10679 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9804bedafae47928d8652278eefb945 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:08:57.193907 10685 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9804bedafae47928d8652278eefb945 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9804bedafae47928d8652278eefb945" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9804bedafae47928d8652278eefb945" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42333 } } }
I20260620 20:08:57.194043 10685 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9804bedafae47928d8652278eefb945 [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:57.194334 10686 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9804bedafae47928d8652278eefb945 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9804bedafae47928d8652278eefb945. Latest consensus state: current_term: 1 leader_uuid: "f9804bedafae47928d8652278eefb945" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9804bedafae47928d8652278eefb945" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42333 } } }
I20260620 20:08:57.194449 10686 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9804bedafae47928d8652278eefb945 [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:57.194413 10693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:08:57.197540 10693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:08:57.205282 10693 catalog_manager.cc:1347] Generated new cluster ID: 9e740d64f820467f85d6138499a8b62d
I20260620 20:08:57.205381 10693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:08:57.221551 10693 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:08:57.222927 10693 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:08:57.242065 10693 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9804bedafae47928d8652278eefb945: Generated new TSK 0
I20260620 20:08:57.242915 10693 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260620 20:08:57.285553 10709 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260620 20:08:57.285686 10709 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260620 20:08:57.286237 10682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:57.257024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45320 (local address 127.5.91.190:42333)
0620 20:08:57.257152 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:57.257157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:57.257371 (+ 214us) server_negotiation.cc:407] Connection header received
0620 20:08:57.258406 (+ 1035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:57.258411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:57.258497 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:57.258646 (+ 149us) server_negotiation.cc:226] Negotiated authn=JWT
0620 20:08:57.258984 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:57.260414 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:57.261865 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:57.262278 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:57.262873 (+ 595us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:57.263572 (+ 699us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 20:08:57.285715 (+ 22143us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36137/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0620 20:08:57.286005 (+ 290us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45320: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36137/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":43}
W20260620 20:08:57.286516 10682 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45320: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36137/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260620 20:08:57.287640 10708 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:08:57.256954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:42333 (local address 127.0.0.1:45320)
0620 20:08:57.257286 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:57.257298 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:57.257821 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:57.258681 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:57.258685 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:57.258697 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0620 20:08:57.258861 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:57.258868 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:57.260880 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:57.260886 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:57.261681 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:57.261698 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:57.262877 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:57.262882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:57.263198 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:57.263274 (+ 76us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 20:08:57.285974 (+ 22700us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36137/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0620 20:08:57.286263 (+ 289us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.91.190:42333: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36137/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":110,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.5.91.190:42333: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36137/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260620 20:08:57.288516 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 10612
2026-06-20T20:08:57Z chronyd exiting
I20260620 20:08:57.326546 5486 test_util.cc:175] -----------------------------------------------
I20260620 20:08:57.326654 5486 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781986111135528-5486-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (369 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260620 20:08:57.328382 5486 mini_oidc.cc:152] Starting JWKS server
I20260620 20:08:57.329444 5486 webserver.cc:460] Webserver started at https://127.0.0.1:39983/ using document root <none> and password file <none>
I20260620 20:08:57.329675 5486 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 20:08:57.330107 5486 webserver.cc:460] Webserver started at http://127.0.0.1:42549/ using document root <none> and password file <none>
2026-06-20T20:08:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:08:57Z Disabled control of system clock
I20260620 20:08:57.340963 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:41015
--webserver_interface=127.5.91.190
--webserver_port=0
--builtin_ntp_servers=127.5.91.148:35897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:41015
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39983/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781986111135528-5486-0/testchainca.cert with env {}
W20260620 20:08:57.464478 10715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:57.464766 10715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:57.464807 10715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:57.469774 10715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:08:57.469862 10715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:57.469878 10715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:57.469894 10715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:08:57.469910 10715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 20:08:57.469925 10715 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 20:08:57.469938 10715 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39983/jwks.json
I20260620 20:08:57.477026 10715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:35897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:41015
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:41015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39983/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781986111135528-5486-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:57.478168 10715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:57.479158 10715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:57.487984 10720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:57.488085 10721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:57.488051 10723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:57.488293 10715 server_base.cc:1034] running on GCE node
I20260620 20:08:57.489133 10715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:57.490432 10715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:57.491623 10715 hybrid_clock.cc:648] HybridClock initialized: now 1781986137491616 us; error 64 us; skew 500 ppm
I20260620 20:08:57.499428 10715 webserver.cc:460] Webserver started at http://127.5.91.190:36629/ using document root <none> and password file <none>
I20260620 20:08:57.500242 10715 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:57.500329 10715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:57.500609 10715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:57.503639 10715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "9e1c9d98a08c47f4b24f9d95f225503e"
format_stamp: "Formatted at 2026-06-20 20:08:57 on dist-test-slave-l8h4"
server_key: "253c8a6984afaebd3b4111fdc883996f"
server_key_iv: "b6e712024fc2ac7c7c6f191fc9d58e0f"
server_key_version: "encryptionkey@0"
I20260620 20:08:57.504300 10715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "9e1c9d98a08c47f4b24f9d95f225503e"
format_stamp: "Formatted at 2026-06-20 20:08:57 on dist-test-slave-l8h4"
server_key: "253c8a6984afaebd3b4111fdc883996f"
server_key_iv: "b6e712024fc2ac7c7c6f191fc9d58e0f"
server_key_version: "encryptionkey@0"
I20260620 20:08:57.508813 10715 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 20:08:57.512626 10729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:57.514236 10715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:08:57.514372 10715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "9e1c9d98a08c47f4b24f9d95f225503e"
format_stamp: "Formatted at 2026-06-20 20:08:57 on dist-test-slave-l8h4"
server_key: "253c8a6984afaebd3b4111fdc883996f"
server_key_iv: "b6e712024fc2ac7c7c6f191fc9d58e0f"
server_key_version: "encryptionkey@0"
I20260620 20:08:57.514626 10715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:57.549106 10715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:57.549850 10715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:57.550019 10715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:57.557448 10715 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:41015
I20260620 20:08:57.557462 10781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:41015 every 8 connection(s)
I20260620 20:08:57.558511 10715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:08:57.559854 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 10715
I20260620 20:08:57.560081 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:08:57.560415 5486 external_mini_cluster.cc:1428] Setting key 0f16a043ae858497116b3bd7e2a9b345
I20260620 20:08:57.562563 10782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:57.568432 10782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e1c9d98a08c47f4b24f9d95f225503e: Bootstrap starting.
I20260620 20:08:57.569837 10785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:57.561863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41082 (local address 127.5.91.190:41015)
0620 20:08:57.562673 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:57.562714 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:57.562751 (+ 37us) server_negotiation.cc:407] Connection header received
0620 20:08:57.563510 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:57.563527 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:57.563915 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:57.564230 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:57.564654 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:57.565636 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:57.566682 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:57.567156 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:57.567302 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:57.568109 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:57.568114 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:57.568126 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:08:57.568508 (+ 382us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:57.568562 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:08:57.568587 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:57.568672 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:57.568845 (+ 173us) server_negotiation.cc:299] Negotiation successful
0620 20:08:57.569108 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":168,"threads_started":1}
I20260620 20:08:57.570751 10782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e1c9d98a08c47f4b24f9d95f225503e: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:57.571593 10782 log.cc:824] T 00000000000000000000000000000000 P 9e1c9d98a08c47f4b24f9d95f225503e: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:57.574319 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:08:57.574924 10782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e1c9d98a08c47f4b24f9d95f225503e: No bootstrap required, opened a new log
I20260620 20:08:57.578933 10782 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e1c9d98a08c47f4b24f9d95f225503e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e1c9d98a08c47f4b24f9d95f225503e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41015 } }
I20260620 20:08:57.579174 10782 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e1c9d98a08c47f4b24f9d95f225503e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:57.579217 10782 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e1c9d98a08c47f4b24f9d95f225503e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e1c9d98a08c47f4b24f9d95f225503e, State: Initialized, Role: FOLLOWER
I20260620 20:08:57.579610 10782 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e1c9d98a08c47f4b24f9d95f225503e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e1c9d98a08c47f4b24f9d95f225503e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41015 } }
I20260620 20:08:57.579728 10782 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e1c9d98a08c47f4b24f9d95f225503e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:08:57.579768 10782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e1c9d98a08c47f4b24f9d95f225503e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:08:57.579829 10782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e1c9d98a08c47f4b24f9d95f225503e [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:57.580953 10782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e1c9d98a08c47f4b24f9d95f225503e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e1c9d98a08c47f4b24f9d95f225503e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41015 } }
I20260620 20:08:57.581254 10782 leader_election.cc:302] T 00000000000000000000000000000000 P 9e1c9d98a08c47f4b24f9d95f225503e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e1c9d98a08c47f4b24f9d95f225503e; no voters:
I20260620 20:08:57.581564 10782 leader_election.cc:288] T 00000000000000000000000000000000 P 9e1c9d98a08c47f4b24f9d95f225503e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:08:57.581677 10787 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e1c9d98a08c47f4b24f9d95f225503e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:57.581899 10787 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e1c9d98a08c47f4b24f9d95f225503e [term 1 LEADER]: Becoming Leader. State: Replica: 9e1c9d98a08c47f4b24f9d95f225503e, State: Running, Role: LEADER
I20260620 20:08:57.582227 10787 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e1c9d98a08c47f4b24f9d95f225503e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e1c9d98a08c47f4b24f9d95f225503e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41015 } }
I20260620 20:08:57.582814 10782 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e1c9d98a08c47f4b24f9d95f225503e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:08:57.583899 10788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e1c9d98a08c47f4b24f9d95f225503e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e1c9d98a08c47f4b24f9d95f225503e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e1c9d98a08c47f4b24f9d95f225503e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41015 } } }
I20260620 20:08:57.583912 10789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e1c9d98a08c47f4b24f9d95f225503e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e1c9d98a08c47f4b24f9d95f225503e. Latest consensus state: current_term: 1 leader_uuid: "9e1c9d98a08c47f4b24f9d95f225503e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e1c9d98a08c47f4b24f9d95f225503e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41015 } } }
I20260620 20:08:57.584043 10789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e1c9d98a08c47f4b24f9d95f225503e [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:57.584041 10788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e1c9d98a08c47f4b24f9d95f225503e [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:57.584497 10796 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:08:57.587352 10796 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:08:57.595458 10796 catalog_manager.cc:1347] Generated new cluster ID: 9e9361d3eff94bedba778ce1c7836326
I20260620 20:08:57.595554 10796 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:08:57.609248 10796 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:08:57.610724 10796 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:08:57.622323 10796 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e1c9d98a08c47f4b24f9d95f225503e: Generated new TSK 0
I20260620 20:08:57.623204 10796 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260620 20:08:57.664950 10812 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260620 20:08:57.665050 10812 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260620 20:08:57.666522 10785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:57.641082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41094 (local address 127.5.91.190:41015)
0620 20:08:57.641217 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:57.641221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:57.641424 (+ 203us) server_negotiation.cc:407] Connection header received
0620 20:08:57.641586 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:57.641590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:57.641651 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:57.641779 (+ 128us) server_negotiation.cc:226] Negotiated authn=JWT
0620 20:08:57.642067 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:57.642709 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:57.644328 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:57.644568 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:57.644648 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:57.645206 (+ 558us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 20:08:57.665305 (+ 20099us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39983/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0620 20:08:57.666307 (+ 1002us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41094: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39983/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":47}
W20260620 20:08:57.666678 10785 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41094: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39983/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260620 20:08:57.667152 10811 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:08:57.640968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41015 (local address 127.0.0.1:41094)
0620 20:08:57.641336 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:57.641348 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:57.641452 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:57.641874 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:57.641876 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:57.641883 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0620 20:08:57.641975 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:57.641980 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:57.642901 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:57.642908 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:57.644161 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:57.644174 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:57.644683 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:57.644687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:57.644777 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:57.644808 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 20:08:57.666188 (+ 21380us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39983/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0620 20:08:57.666965 (+ 777us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.91.190:41015: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39983/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":110,"threads_started":1}
I20260620 20:08:57.668111 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 10715
2026-06-20T20:08:57Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (380 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260620 20:08:57.707921 5486 mini_oidc.cc:152] Starting JWKS server
I20260620 20:08:57.709256 5486 webserver.cc:460] Webserver started at https://127.0.0.1:33667/ using document root <none> and password file <none>
I20260620 20:08:57.709401 5486 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 20:08:57.709688 5486 webserver.cc:460] Webserver started at http://127.0.0.1:36775/ using document root <none> and password file <none>
2026-06-20T20:08:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:08:57Z Disabled control of system clock
I20260620 20:08:57.720727 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:33585
--webserver_interface=127.5.91.190
--webserver_port=0
--builtin_ntp_servers=127.5.91.148:44649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:33585
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33667/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260620 20:08:57.853652 10818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:57.853965 10818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:57.854008 10818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:57.858773 10818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:08:57.858860 10818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:57.858876 10818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:57.858892 10818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:08:57.858906 10818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 20:08:57.858922 10818 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 20:08:57.858934 10818 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33667/jwks.json
I20260620 20:08:57.864984 10818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:44649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:33585
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:33585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33667/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:57.866297 10818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:57.867797 10818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:57.881443 10824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:57.881438 10823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:57.881939 10818 server_base.cc:1034] running on GCE node
W20260620 20:08:57.881971 10826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:57.882714 10818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:57.883944 10818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:57.885147 10818 hybrid_clock.cc:648] HybridClock initialized: now 1781986137885112 us; error 50 us; skew 500 ppm
I20260620 20:08:57.887454 10818 webserver.cc:460] Webserver started at http://127.5.91.190:37655/ using document root <none> and password file <none>
I20260620 20:08:57.888063 10818 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:57.888173 10818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:57.888408 10818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:57.890618 10818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "c698ad1726664b88bbebdb187db7f23e"
format_stamp: "Formatted at 2026-06-20 20:08:57 on dist-test-slave-l8h4"
server_key: "f3cf186af00607af1f67cbf8a61dd1fd"
server_key_iv: "58ba916a1378d251d0428ecd34e8cb51"
server_key_version: "encryptionkey@0"
I20260620 20:08:57.891155 10818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "c698ad1726664b88bbebdb187db7f23e"
format_stamp: "Formatted at 2026-06-20 20:08:57 on dist-test-slave-l8h4"
server_key: "f3cf186af00607af1f67cbf8a61dd1fd"
server_key_iv: "58ba916a1378d251d0428ecd34e8cb51"
server_key_version: "encryptionkey@0"
I20260620 20:08:57.895728 10818 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 20:08:57.898700 10832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:57.900476 10818 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:08:57.900615 10818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "c698ad1726664b88bbebdb187db7f23e"
format_stamp: "Formatted at 2026-06-20 20:08:57 on dist-test-slave-l8h4"
server_key: "f3cf186af00607af1f67cbf8a61dd1fd"
server_key_iv: "58ba916a1378d251d0428ecd34e8cb51"
server_key_version: "encryptionkey@0"
I20260620 20:08:57.901414 10818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:57.924198 10818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:57.924929 10818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:57.925091 10818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:57.933346 10818 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:33585
I20260620 20:08:57.933369 10884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:33585 every 8 connection(s)
I20260620 20:08:57.934424 10818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:08:57.938630 10885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:57.939302 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 10818
I20260620 20:08:57.939414 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:08:57.939671 5486 external_mini_cluster.cc:1428] Setting key d9e53240da2c2d85354de1d28c37fbd7
I20260620 20:08:57.945188 10885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c698ad1726664b88bbebdb187db7f23e: Bootstrap starting.
I20260620 20:08:57.949345 10885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c698ad1726664b88bbebdb187db7f23e: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:57.949568 10888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:57.941344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51744 (local address 127.5.91.190:33585)
0620 20:08:57.942090 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:57.942107 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:57.942180 (+ 73us) server_negotiation.cc:407] Connection header received
0620 20:08:57.942888 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:57.942906 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:57.943303 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:57.943622 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:57.944048 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:57.945010 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:57.945914 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:57.946582 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:57.946743 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:57.947616 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:57.947621 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:57.947636 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:08:57.948142 (+ 506us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:57.948210 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:08:57.948232 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:57.948343 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:57.948486 (+ 143us) server_negotiation.cc:299] Negotiation successful
0620 20:08:57.948754 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":155,"threads_started":1}
I20260620 20:08:57.950327 10885 log.cc:824] T 00000000000000000000000000000000 P c698ad1726664b88bbebdb187db7f23e: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:57.953143 10885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c698ad1726664b88bbebdb187db7f23e: No bootstrap required, opened a new log
I20260620 20:08:57.954057 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:08:57.958251 10885 raft_consensus.cc:348] T 00000000000000000000000000000000 P c698ad1726664b88bbebdb187db7f23e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c698ad1726664b88bbebdb187db7f23e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33585 } }
I20260620 20:08:57.958758 10885 raft_consensus.cc:374] T 00000000000000000000000000000000 P c698ad1726664b88bbebdb187db7f23e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:57.958807 10885 raft_consensus.cc:729] T 00000000000000000000000000000000 P c698ad1726664b88bbebdb187db7f23e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c698ad1726664b88bbebdb187db7f23e, State: Initialized, Role: FOLLOWER
I20260620 20:08:57.959522 10885 consensus_queue.cc:260] T 00000000000000000000000000000000 P c698ad1726664b88bbebdb187db7f23e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c698ad1726664b88bbebdb187db7f23e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33585 } }
I20260620 20:08:57.959790 10885 raft_consensus.cc:388] T 00000000000000000000000000000000 P c698ad1726664b88bbebdb187db7f23e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:08:57.959861 10885 raft_consensus.cc:482] T 00000000000000000000000000000000 P c698ad1726664b88bbebdb187db7f23e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:08:57.960103 10885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c698ad1726664b88bbebdb187db7f23e [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:57.961273 10885 raft_consensus.cc:504] T 00000000000000000000000000000000 P c698ad1726664b88bbebdb187db7f23e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c698ad1726664b88bbebdb187db7f23e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33585 } }
I20260620 20:08:57.961688 10885 leader_election.cc:302] T 00000000000000000000000000000000 P c698ad1726664b88bbebdb187db7f23e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c698ad1726664b88bbebdb187db7f23e; no voters:
I20260620 20:08:57.962201 10885 leader_election.cc:288] T 00000000000000000000000000000000 P c698ad1726664b88bbebdb187db7f23e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:08:57.962353 10890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c698ad1726664b88bbebdb187db7f23e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:57.962625 10890 raft_consensus.cc:686] T 00000000000000000000000000000000 P c698ad1726664b88bbebdb187db7f23e [term 1 LEADER]: Becoming Leader. State: Replica: c698ad1726664b88bbebdb187db7f23e, State: Running, Role: LEADER
I20260620 20:08:57.963071 10890 consensus_queue.cc:237] T 00000000000000000000000000000000 P c698ad1726664b88bbebdb187db7f23e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c698ad1726664b88bbebdb187db7f23e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33585 } }
I20260620 20:08:57.964145 10885 sys_catalog.cc:564] T 00000000000000000000000000000000 P c698ad1726664b88bbebdb187db7f23e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:08:57.965133 10892 sys_catalog.cc:455] T 00000000000000000000000000000000 P c698ad1726664b88bbebdb187db7f23e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c698ad1726664b88bbebdb187db7f23e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c698ad1726664b88bbebdb187db7f23e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33585 } } }
I20260620 20:08:57.965444 10892 sys_catalog.cc:458] T 00000000000000000000000000000000 P c698ad1726664b88bbebdb187db7f23e [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:57.965775 10891 sys_catalog.cc:455] T 00000000000000000000000000000000 P c698ad1726664b88bbebdb187db7f23e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c698ad1726664b88bbebdb187db7f23e. Latest consensus state: current_term: 1 leader_uuid: "c698ad1726664b88bbebdb187db7f23e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c698ad1726664b88bbebdb187db7f23e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33585 } } }
I20260620 20:08:57.965885 10891 sys_catalog.cc:458] T 00000000000000000000000000000000 P c698ad1726664b88bbebdb187db7f23e [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:57.965983 10899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:08:57.969334 10899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:08:57.976701 10899 catalog_manager.cc:1347] Generated new cluster ID: 6151fb89d8b542e09df20d26b863a37a
I20260620 20:08:57.976801 10899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:08:57.996461 10899 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:08:57.997989 10899 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:08:58.024065 10899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c698ad1726664b88bbebdb187db7f23e: Generated new TSK 0
I20260620 20:08:58.025004 10899 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:08:58.039307 10888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:58.033262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51756 (local address 127.5.91.190:33585)
0620 20:08:58.033391 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:58.033395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:58.033493 (+ 98us) server_negotiation.cc:407] Connection header received
0620 20:08:58.033674 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:58.033676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:58.033730 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:58.033825 (+ 95us) server_negotiation.cc:226] Negotiated authn=JWT
0620 20:08:58.034099 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:58.035216 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:58.036737 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:58.037052 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:58.037211 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:58.037627 (+ 416us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 20:08:58.038922 (+ 1295us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33667/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0620 20:08:58.039162 (+ 240us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51756: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33667/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":40}
W20260620 20:08:58.039392 10888 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51756: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33667/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260620 20:08:58.039395 10914 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:08:58.033086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33585 (local address 127.0.0.1:51756)
0620 20:08:58.033367 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:58.033383 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:58.033512 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:58.033833 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:58.033836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:58.033843 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0620 20:08:58.033965 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:58.033972 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:58.035367 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:58.035371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:58.036437 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:58.036448 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:58.037249 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:58.037253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:58.037350 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:58.037385 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 20:08:58.039089 (+ 1704us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33667/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0620 20:08:58.039236 (+ 147us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.91.190:33585: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33667/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":97,"threads_started":1}
I20260620 20:08:58.039779 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 10818
E20260620 20:08:58.052446 10915 webserver.cc:533] Webserver: sslize failed:
E20260620 20:08:58.052807 10915 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-20T20:08:58Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (373 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-20T20:08:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:08:58Z Disabled control of system clock
I20260620 20:08:58.091106 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:43953
--webserver_interface=127.5.91.190
--webserver_port=0
--builtin_ntp_servers=127.5.91.148:33575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:43953
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781986111135528-5486-0/testchainca.cert with env {}
W20260620 20:08:58.226503 10919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:08:58.226962 10919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:08:58.227020 10919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:08:58.231686 10919 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:08:58.231773 10919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:08:58.231789 10919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:08:58.231805 10919 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:08:58.231822 10919 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 20:08:58.231837 10919 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 20:08:58.231849 10919 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260620 20:08:58.240240 10919 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:33575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:43953
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:43953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781986111135528-5486-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:08:58.241446 10919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:08:58.242533 10919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:08:58.252705 10927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:58.253086 10919 server_base.cc:1034] running on GCE node
W20260620 20:08:58.252852 10925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:08:58.252852 10924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:08:58.253932 10919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:08:58.255184 10919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:08:58.256389 10919 hybrid_clock.cc:648] HybridClock initialized: now 1781986138256344 us; error 61 us; skew 500 ppm
I20260620 20:08:58.258638 10919 webserver.cc:460] Webserver started at http://127.5.91.190:43649/ using document root <none> and password file <none>
I20260620 20:08:58.259307 10919 fs_manager.cc:359] Metadata directory not provided
I20260620 20:08:58.259389 10919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:08:58.259583 10919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:08:58.261823 10919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "f08a507ac78246e9b92528c286651a5f"
format_stamp: "Formatted at 2026-06-20 20:08:58 on dist-test-slave-l8h4"
server_key: "de7ab0b65260b31095142c0c45c4a4f0"
server_key_iv: "74dab06c706cda8eb40e915e6073ee7a"
server_key_version: "encryptionkey@0"
I20260620 20:08:58.262686 10919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "f08a507ac78246e9b92528c286651a5f"
format_stamp: "Formatted at 2026-06-20 20:08:58 on dist-test-slave-l8h4"
server_key: "de7ab0b65260b31095142c0c45c4a4f0"
server_key_iv: "74dab06c706cda8eb40e915e6073ee7a"
server_key_version: "encryptionkey@0"
I20260620 20:08:58.267330 10919 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260620 20:08:58.270385 10933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:08:58.272511 10919 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260620 20:08:58.272668 10919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "f08a507ac78246e9b92528c286651a5f"
format_stamp: "Formatted at 2026-06-20 20:08:58 on dist-test-slave-l8h4"
server_key: "de7ab0b65260b31095142c0c45c4a4f0"
server_key_iv: "74dab06c706cda8eb40e915e6073ee7a"
server_key_version: "encryptionkey@0"
I20260620 20:08:58.272800 10919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:08:58.296262 10919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:08:58.297042 10919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:08:58.297230 10919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:08:58.305302 10919 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:43953
I20260620 20:08:58.305346 10985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:43953 every 8 connection(s)
I20260620 20:08:58.306521 10919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:08:58.308979 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 10919
I20260620 20:08:58.309103 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:08:58.309362 5486 external_mini_cluster.cc:1428] Setting key f4509a9c784a993abf3e06266fee8eda
I20260620 20:08:58.309849 10986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:08:58.319887 10986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f08a507ac78246e9b92528c286651a5f: Bootstrap starting.
I20260620 20:08:58.323201 10986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f08a507ac78246e9b92528c286651a5f: Neither blocks nor log segments found. Creating new log.
I20260620 20:08:58.324054 10986 log.cc:824] T 00000000000000000000000000000000 P f08a507ac78246e9b92528c286651a5f: Log is configured to *not* fsync() on all Append() calls
I20260620 20:08:58.324888 10989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:58.310978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53700 (local address 127.5.91.190:43953)
0620 20:08:58.312822 (+ 1844us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:58.313000 (+ 178us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:58.313140 (+ 140us) server_negotiation.cc:407] Connection header received
0620 20:08:58.314817 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:58.314855 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:58.315573 (+ 718us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:58.316145 (+ 572us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:08:58.316847 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:58.318425 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:58.319912 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:58.320603 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:58.320758 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:58.322059 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:08:58.322065 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:08:58.322080 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:08:58.322914 (+ 834us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:08:58.323012 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:08:58.323056 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:08:58.323182 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0620 20:08:58.323477 (+ 295us) server_negotiation.cc:299] Negotiation successful
0620 20:08:58.323788 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":743,"thread_start_us":154,"threads_started":1}
I20260620 20:08:58.326421 10986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f08a507ac78246e9b92528c286651a5f: No bootstrap required, opened a new log
I20260620 20:08:58.328613 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:08:58.331672 10986 raft_consensus.cc:348] T 00000000000000000000000000000000 P f08a507ac78246e9b92528c286651a5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f08a507ac78246e9b92528c286651a5f" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 43953 } }
I20260620 20:08:58.332000 10986 raft_consensus.cc:374] T 00000000000000000000000000000000 P f08a507ac78246e9b92528c286651a5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:08:58.332059 10986 raft_consensus.cc:729] T 00000000000000000000000000000000 P f08a507ac78246e9b92528c286651a5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f08a507ac78246e9b92528c286651a5f, State: Initialized, Role: FOLLOWER
I20260620 20:08:58.332544 10986 consensus_queue.cc:260] T 00000000000000000000000000000000 P f08a507ac78246e9b92528c286651a5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f08a507ac78246e9b92528c286651a5f" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 43953 } }
I20260620 20:08:58.332674 10986 raft_consensus.cc:388] T 00000000000000000000000000000000 P f08a507ac78246e9b92528c286651a5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:08:58.332731 10986 raft_consensus.cc:482] T 00000000000000000000000000000000 P f08a507ac78246e9b92528c286651a5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:08:58.332830 10986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f08a507ac78246e9b92528c286651a5f [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:08:58.334074 10986 raft_consensus.cc:504] T 00000000000000000000000000000000 P f08a507ac78246e9b92528c286651a5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f08a507ac78246e9b92528c286651a5f" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 43953 } }
I20260620 20:08:58.334450 10986 leader_election.cc:302] T 00000000000000000000000000000000 P f08a507ac78246e9b92528c286651a5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f08a507ac78246e9b92528c286651a5f; no voters:
I20260620 20:08:58.334760 10986 leader_election.cc:288] T 00000000000000000000000000000000 P f08a507ac78246e9b92528c286651a5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:08:58.334921 10991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f08a507ac78246e9b92528c286651a5f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:08:58.335170 10991 raft_consensus.cc:686] T 00000000000000000000000000000000 P f08a507ac78246e9b92528c286651a5f [term 1 LEADER]: Becoming Leader. State: Replica: f08a507ac78246e9b92528c286651a5f, State: Running, Role: LEADER
I20260620 20:08:58.335479 10991 consensus_queue.cc:237] T 00000000000000000000000000000000 P f08a507ac78246e9b92528c286651a5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f08a507ac78246e9b92528c286651a5f" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 43953 } }
I20260620 20:08:58.335996 10986 sys_catalog.cc:564] T 00000000000000000000000000000000 P f08a507ac78246e9b92528c286651a5f [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:08:58.337010 10993 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08a507ac78246e9b92528c286651a5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f08a507ac78246e9b92528c286651a5f. Latest consensus state: current_term: 1 leader_uuid: "f08a507ac78246e9b92528c286651a5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f08a507ac78246e9b92528c286651a5f" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 43953 } } }
I20260620 20:08:58.337165 10993 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08a507ac78246e9b92528c286651a5f [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:58.338268 11000 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:08:58.338075 10992 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08a507ac78246e9b92528c286651a5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f08a507ac78246e9b92528c286651a5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f08a507ac78246e9b92528c286651a5f" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 43953 } } }
I20260620 20:08:58.339131 10992 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08a507ac78246e9b92528c286651a5f [sys.catalog]: This master's current role is: LEADER
I20260620 20:08:58.343284 11000 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:08:58.351276 11000 catalog_manager.cc:1347] Generated new cluster ID: b8efbe650bf24754806154c8a268e15a
I20260620 20:08:58.351423 11000 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:08:58.370313 11000 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:08:58.372148 11000 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:08:58.378841 11000 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f08a507ac78246e9b92528c286651a5f: Generated new TSK 0
I20260620 20:08:58.379809 11000 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:08:58.868688 10989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:08:58.393149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53706 (local address 127.5.91.190:43953)
0620 20:08:58.393286 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0620 20:08:58.393291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:08:58.393310 (+ 19us) server_negotiation.cc:407] Connection header received
0620 20:08:58.393559 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:08:58.393563 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:08:58.393633 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:08:58.393767 (+ 134us) server_negotiation.cc:226] Negotiated authn=JWT
0620 20:08:58.394095 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:58.395100 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:58.396123 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:08:58.396482 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:08:58.396719 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:58.397030 (+ 311us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 20:08:58.868241 (+471211us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0620 20:08:58.868565 (+ 324us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53706: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":59}
W20260620 20:08:58.868759 10989 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53706: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260620 20:08:58.868873 11015 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:08:58.392970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:43953 (local address 127.0.0.1:53706)
0620 20:08:58.393227 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0620 20:08:58.393240 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:08:58.393332 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:08:58.393866 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:08:58.393869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:08:58.393877 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0620 20:08:58.393984 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:58.393989 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:58.395328 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:58.395331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:58.395997 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:08:58.396006 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:08:58.396624 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:08:58.396627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:08:58.396704 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:08:58.396732 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 20:08:58.868499 (+471767us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0620 20:08:58.868681 (+ 182us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.91.190:43953: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":91,"threads_started":1}
I20260620 20:08:58.869361 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 10919
2026-06-20T20:08:58Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (807 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (139 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (149 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:08:59 dist-test-slave-l8h4 krb5kdc[11027](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:08:59 dist-test-slave-l8h4 krb5kdc[11027](info): set up 2 sockets
Jun 20 20:08:59 dist-test-slave-l8h4 krb5kdc[11027](info): commencing operation
krb5kdc: starting...
W20260620 20:09:00.247434 5486 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:09:00 dist-test-slave-l8h4 krb5kdc[11027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:09:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:09:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:09:00.391167 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:34995
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:34797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:34995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:09:00.517584 11040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:00.517877 11040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:00.517935 11040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:00.522123 11040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:09:00.522218 11040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:00.522245 11040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:00.522266 11040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:09:00.522286 11040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:09:00.529721 11040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:34797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:34995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:34995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:00.531344 11040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:00.532753 11040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:00.542399 11046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:00.542399 11045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:00.542730 11040 server_base.cc:1034] running on GCE node
W20260620 20:09:00.542799 11048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:00.543462 11040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:00.544801 11040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:00.546077 11040 hybrid_clock.cc:648] HybridClock initialized: now 1781986140546028 us; error 56 us; skew 500 ppm
Jun 20 20:09:00 dist-test-slave-l8h4 krb5kdc[11027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986140, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:00.549556 11040 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:09:00.550606 11040 webserver.cc:460] Webserver started at http://127.5.91.190:46839/ using document root <none> and password file <none>
I20260620 20:09:00.551671 11040 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:00.551802 11040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:00.552096 11040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:00.554936 11040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "a9a2697e811a4b429aa841e5df5d423e"
format_stamp: "Formatted at 2026-06-20 20:09:00 on dist-test-slave-l8h4"
server_key: "11aace5d23ad6d8387139fe0eaa27e01"
server_key_iv: "49c742f9992fb0d54fa31d4436689b3f"
server_key_version: "encryptionkey@0"
I20260620 20:09:00.555577 11040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "a9a2697e811a4b429aa841e5df5d423e"
format_stamp: "Formatted at 2026-06-20 20:09:00 on dist-test-slave-l8h4"
server_key: "11aace5d23ad6d8387139fe0eaa27e01"
server_key_iv: "49c742f9992fb0d54fa31d4436689b3f"
server_key_version: "encryptionkey@0"
I20260620 20:09:00.560109 11040 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260620 20:09:00.562920 11055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:00.564316 11040 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260620 20:09:00.564456 11040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "a9a2697e811a4b429aa841e5df5d423e"
format_stamp: "Formatted at 2026-06-20 20:09:00 on dist-test-slave-l8h4"
server_key: "11aace5d23ad6d8387139fe0eaa27e01"
server_key_iv: "49c742f9992fb0d54fa31d4436689b3f"
server_key_version: "encryptionkey@0"
I20260620 20:09:00.564575 11040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:00.578570 11040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:00.583097 11040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:00.583478 11040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:00.591225 11040 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:34995
I20260620 20:09:00.591344 11107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:34995 every 8 connection(s)
I20260620 20:09:00.592216 11040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:09:00.595235 11108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:00.598719 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 11040
I20260620 20:09:00.598866 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:09:00.599128 5486 external_mini_cluster.cc:1428] Setting key 3b80e477098747a9ad39b5cac088542b
I20260620 20:09:00.601579 11108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9a2697e811a4b429aa841e5df5d423e: Bootstrap starting.
I20260620 20:09:00.604400 11108 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9a2697e811a4b429aa841e5df5d423e: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:00.605254 11108 log.cc:824] T 00000000000000000000000000000000 P a9a2697e811a4b429aa841e5df5d423e: Log is configured to *not* fsync() on all Append() calls
I20260620 20:09:00.607402 11108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9a2697e811a4b429aa841e5df5d423e: No bootstrap required, opened a new log
Jun 20 20:09:00 dist-test-slave-l8h4 krb5kdc[11027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:09:00.611665 11108 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9a2697e811a4b429aa841e5df5d423e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9a2697e811a4b429aa841e5df5d423e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 34995 } }
I20260620 20:09:00.611974 11108 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9a2697e811a4b429aa841e5df5d423e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:00.612016 11108 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9a2697e811a4b429aa841e5df5d423e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a2697e811a4b429aa841e5df5d423e, State: Initialized, Role: FOLLOWER
I20260620 20:09:00.612460 11108 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9a2697e811a4b429aa841e5df5d423e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9a2697e811a4b429aa841e5df5d423e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 34995 } }
I20260620 20:09:00.612586 11108 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9a2697e811a4b429aa841e5df5d423e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:09:00.612633 11108 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9a2697e811a4b429aa841e5df5d423e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:09:00.612730 11108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9a2697e811a4b429aa841e5df5d423e [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:00.613852 11108 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9a2697e811a4b429aa841e5df5d423e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9a2697e811a4b429aa841e5df5d423e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 34995 } }
I20260620 20:09:00.614212 11108 leader_election.cc:302] T 00000000000000000000000000000000 P a9a2697e811a4b429aa841e5df5d423e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9a2697e811a4b429aa841e5df5d423e; no voters:
I20260620 20:09:00.614570 11108 leader_election.cc:288] T 00000000000000000000000000000000 P a9a2697e811a4b429aa841e5df5d423e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:09:00.614642 11113 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9a2697e811a4b429aa841e5df5d423e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:09:00.614928 11113 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9a2697e811a4b429aa841e5df5d423e [term 1 LEADER]: Becoming Leader. State: Replica: a9a2697e811a4b429aa841e5df5d423e, State: Running, Role: LEADER
I20260620 20:09:00.615253 11113 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9a2697e811a4b429aa841e5df5d423e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9a2697e811a4b429aa841e5df5d423e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 34995 } }
I20260620 20:09:00.615908 11108 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9a2697e811a4b429aa841e5df5d423e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:09:00.616231 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:00.600670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41128 (local address 127.5.91.190:34995)
0620 20:09:00.601130 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:00.601148 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:00.601191 (+ 43us) server_negotiation.cc:407] Connection header received
0620 20:09:00.602030 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:00.602053 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:00.602518 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:00.602918 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:00.603577 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:00.605018 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:00.606310 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:00.606986 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:00.607149 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:00.610472 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:00.610487 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:00.610632 (+ 145us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:00.610670 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:00.612767 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:00.613304 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:00.613308 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:00.613313 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:00.613478 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:00.613763 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:00.613766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:00.613767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:00.614264 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:00.614477 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:00.614785 (+ 308us) server_negotiation.cc:299] Negotiation successful
0620 20:09:00.615263 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":141,"threads_started":1}
I20260620 20:09:00.616523 11113 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9a2697e811a4b429aa841e5df5d423e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9a2697e811a4b429aa841e5df5d423e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9a2697e811a4b429aa841e5df5d423e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 34995 } } }
I20260620 20:09:00.616645 11113 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9a2697e811a4b429aa841e5df5d423e [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:00.616829 11113 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9a2697e811a4b429aa841e5df5d423e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9a2697e811a4b429aa841e5df5d423e. Latest consensus state: current_term: 1 leader_uuid: "a9a2697e811a4b429aa841e5df5d423e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9a2697e811a4b429aa841e5df5d423e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 34995 } } }
I20260620 20:09:00.616918 11113 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9a2697e811a4b429aa841e5df5d423e [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:00.617116 11120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:09:00.620569 11120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:09:00.626879 11120 catalog_manager.cc:1347] Generated new cluster ID: 5fdf9f34693d40758f1a6e519e4b0038
I20260620 20:09:00.626965 11120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:09:00.643182 11120 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:09:00.645056 11120 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:09:00.655583 11120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9a2697e811a4b429aa841e5df5d423e: Generated new TSK 0
I20260620 20:09:00.656504 11120 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.91.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
I20260620 20:09:00.696233 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:34995
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:34797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:09:00.843415 11134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:00.843827 11134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:00.843880 11134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:00.848811 11134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:00.848922 11134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:00.848994 11134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:09:00.856043 11134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:34797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:34995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:00.857168 11134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:00.858204 11134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:00.868839 11142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:00.868950 11139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:00.868992 11140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:00.869441 11134 server_base.cc:1034] running on GCE node
I20260620 20:09:00.869843 11134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:00.870996 11134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:00.872246 11134 hybrid_clock.cc:648] HybridClock initialized: now 1781986140872217 us; error 51 us; skew 500 ppm
Jun 20 20:09:00 dist-test-slave-l8h4 krb5kdc[11027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986140, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:00.875473 11134 init.cc:377] Logged in from keytab as kudu/127.5.91.129@KRBTEST.COM (short username kudu)
I20260620 20:09:00.876464 11134 webserver.cc:460] Webserver started at http://127.5.91.129:42871/ using document root <none> and password file <none>
I20260620 20:09:00.877027 11134 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:00.877103 11134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:00.877295 11134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:00.879453 11134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "d7a5a793eefc4b8c9e41952209d18ae4"
format_stamp: "Formatted at 2026-06-20 20:09:00 on dist-test-slave-l8h4"
server_key: "4f74198247589a4f9812d75e94246f53"
server_key_iv: "78e73723238731fe6e2c04c34d2fbc98"
server_key_version: "encryptionkey@0"
I20260620 20:09:00.880100 11134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "d7a5a793eefc4b8c9e41952209d18ae4"
format_stamp: "Formatted at 2026-06-20 20:09:00 on dist-test-slave-l8h4"
server_key: "4f74198247589a4f9812d75e94246f53"
server_key_iv: "78e73723238731fe6e2c04c34d2fbc98"
server_key_version: "encryptionkey@0"
I20260620 20:09:00.885540 11134 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260620 20:09:00.889602 11149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:00.891232 11134 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:09:00.891367 11134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "d7a5a793eefc4b8c9e41952209d18ae4"
format_stamp: "Formatted at 2026-06-20 20:09:00 on dist-test-slave-l8h4"
server_key: "4f74198247589a4f9812d75e94246f53"
server_key_iv: "78e73723238731fe6e2c04c34d2fbc98"
server_key_version: "encryptionkey@0"
I20260620 20:09:00.891500 11134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:00.912352 11134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:00.916340 11134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:00.916556 11134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:00.917178 11134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:00.918663 11134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:00.918742 11134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:00.918818 11134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:00.918848 11134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:00.929466 11134 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:37511
I20260620 20:09:00.929529 11262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:37511 every 8 connection(s)
I20260620 20:09:00.930446 11134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:09:00.936043 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 11134
I20260620 20:09:00.936192 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:09:00.936484 5486 external_mini_cluster.cc:1428] Setting key 655e33a86d72b065b238fd74be0e4579
Jun 20 20:09:00 dist-test-slave-l8h4 krb5kdc[11027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986140, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.91.130@KRBTEST.COM; defaulting to no policy
I20260620 20:09:00.948236 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:00.932791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:49901 (local address 127.5.91.190:34995)
0620 20:09:00.932936 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:00.932940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:00.934597 (+ 1657us) server_negotiation.cc:407] Connection header received
0620 20:09:00.936333 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:00.936337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:00.936394 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:00.936477 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:00.937906 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:00.938614 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:00.939825 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:00.940186 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:00.940290 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:00.943435 (+ 3145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:00.943440 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:00.943443 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:00.943474 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:00.945120 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:00.945675 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:00.945679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:00.945680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:00.945765 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:00.946604 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:00.946606 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:00.946608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:00.946820 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:00.946921 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:00.947859 (+ 938us) server_negotiation.cc:299] Negotiation successful
0620 20:09:00.948058 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 20:09:00.948882 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:00.933418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:34995 (local address 127.5.91.129:49901)
0620 20:09:00.934304 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:00.934363 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:00.935986 (+ 1623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:00.936700 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:00.936721 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:00.937139 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:00.937760 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:00.937780 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:00.938778 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:00.938781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:00.939694 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:00.939704 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:00.940362 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:00.940366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:00.940738 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:00.941556 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:00.941573 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:00.943236 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:00.945256 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:00.945261 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:00.945275 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:00.945568 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:00.945971 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:00.945975 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:00.945977 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:00.946367 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:00.946997 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:00.947003 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:00.947386 (+ 383us) client_negotiation.cc:769] Sending connection context
0620 20:09:00.947575 (+ 189us) client_negotiation.cc:240] Negotiation successful
0620 20:09:00.947883 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":283,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.130@KRBTEST.COM" created.
I20260620 20:09:00.950690 11263 heartbeater.cc:347] Connected to a master server at 127.5.91.190:34995
I20260620 20:09:00.951120 11263 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:00.951833 11263 heartbeater.cc:511] Master 127.5.91.190:34995 requested a full tablet report, sending...
I20260620 20:09:00.953948 11072 ts_manager.cc:194] Registered new tserver with Master: d7a5a793eefc4b8c9e41952209d18ae4 (127.5.91.129:37511)
I20260620 20:09:00.955228 11072 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:49901
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
I20260620 20:09:00.967233 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:34995
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:34797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:09:01.105803 11268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:01.106096 11268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:01.106175 11268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:01.110437 11268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:01.110514 11268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:01.110582 11268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:09:01.118343 11268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:34797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:34995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:01.120026 11268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:01.121266 11268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:01.132012 11273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:01.132057 11274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:01.132006 11276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:01.132465 11268 server_base.cc:1034] running on GCE node
I20260620 20:09:01.132929 11268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:01.133633 11268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:01.134945 11268 hybrid_clock.cc:648] HybridClock initialized: now 1781986141134922 us; error 53 us; skew 500 ppm
Jun 20 20:09:01 dist-test-slave-l8h4 krb5kdc[11027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986141, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:01.138902 11268 init.cc:377] Logged in from keytab as kudu/127.5.91.130@KRBTEST.COM (short username kudu)
I20260620 20:09:01.140348 11268 webserver.cc:460] Webserver started at http://127.5.91.130:37969/ using document root <none> and password file <none>
I20260620 20:09:01.140973 11268 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:01.141042 11268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:01.141232 11268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:01.143370 11268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "5e30b49eccde4bbb9ade33bb88e2c3c2"
format_stamp: "Formatted at 2026-06-20 20:09:01 on dist-test-slave-l8h4"
server_key: "ce0fc04a93fba8fb0074cc34b917f61e"
server_key_iv: "43d1df7fad6d4cd52f96f6cf23baceb6"
server_key_version: "encryptionkey@0"
I20260620 20:09:01.143919 11268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "5e30b49eccde4bbb9ade33bb88e2c3c2"
format_stamp: "Formatted at 2026-06-20 20:09:01 on dist-test-slave-l8h4"
server_key: "ce0fc04a93fba8fb0074cc34b917f61e"
server_key_iv: "43d1df7fad6d4cd52f96f6cf23baceb6"
server_key_version: "encryptionkey@0"
I20260620 20:09:01.147920 11268 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 20:09:01.150513 11283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:01.152118 11268 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:09:01.152244 11268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "5e30b49eccde4bbb9ade33bb88e2c3c2"
format_stamp: "Formatted at 2026-06-20 20:09:01 on dist-test-slave-l8h4"
server_key: "ce0fc04a93fba8fb0074cc34b917f61e"
server_key_iv: "43d1df7fad6d4cd52f96f6cf23baceb6"
server_key_version: "encryptionkey@0"
I20260620 20:09:01.152352 11268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:01.167677 11268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:01.171309 11268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:01.171511 11268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:01.172150 11268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:01.173003 11268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:01.173050 11268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:01.173090 11268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:01.173105 11268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:01.184254 11268 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:33999
I20260620 20:09:01.184329 11396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:33999 every 8 connection(s)
I20260620 20:09:01.185289 11268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:09:01.185832 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 11268
I20260620 20:09:01.185940 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:09:01.186228 5486 external_mini_cluster.cc:1428] Setting key e425ea60b9d182d12a5ee61e933ddc34
WARNING: no policy specified for kudu/127.5.91.131@KRBTEST.COM; defaulting to no policy
Jun 20 20:09:01 dist-test-slave-l8h4 krb5kdc[11027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986141, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.131@KRBTEST.COM" created.
I20260620 20:09:01.203291 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:01.187457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:33613 (local address 127.5.91.190:34995)
0620 20:09:01.187562 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:01.187566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:01.188622 (+ 1056us) server_negotiation.cc:407] Connection header received
0620 20:09:01.190207 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:01.190211 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:01.190269 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:01.190340 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:01.191624 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:01.192189 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:01.194017 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:01.194282 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:01.194368 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:01.197391 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:01.197395 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:01.197398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:01.197426 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:01.199961 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:01.200722 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:01.200727 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:01.200729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:01.200869 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:01.201552 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:01.201557 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:01.201558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:01.201830 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:01.202103 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:01.202982 (+ 879us) server_negotiation.cc:299] Negotiation successful
0620 20:09:01.203126 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 20:09:01.205063 11399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:01.187900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:34995 (local address 127.5.91.130:33613)
0620 20:09:01.188441 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:01.188499 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:01.189959 (+ 1460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:01.190540 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:01.190548 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:01.190923 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:01.191470 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:01.191485 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:01.192425 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:01.192429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:01.193863 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:01.193877 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:01.194450 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:01.194455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:01.194688 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:01.195368 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:01.195387 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:01.197125 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:01.200104 (+ 2979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:01.200112 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:01.200132 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:01.200530 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:01.201166 (+ 636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:01.201170 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:01.201172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:01.201414 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:01.202106 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:01.202117 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:01.202597 (+ 480us) client_negotiation.cc:769] Sending connection context
0620 20:09:01.202873 (+ 276us) client_negotiation.cc:240] Negotiation successful
0620 20:09:01.203363 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":121,"threads_started":1}
I20260620 20:09:01.206461 11397 heartbeater.cc:347] Connected to a master server at 127.5.91.190:34995
I20260620 20:09:01.206749 11397 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:01.207397 11397 heartbeater.cc:511] Master 127.5.91.190:34995 requested a full tablet report, sending...
I20260620 20:09:01.208976 11072 ts_manager.cc:194] Registered new tserver with Master: 5e30b49eccde4bbb9ade33bb88e2c3c2 (127.5.91.130:33999)
I20260620 20:09:01.209774 11072 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:33613
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
I20260620 20:09:01.224354 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:34995
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:34797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:09:01.364856 11402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:01.365217 11402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:01.365283 11402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:01.370347 11402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:01.370432 11402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:01.370519 11402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:09:01.376689 11402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:34797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:34995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:01.377887 11402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:01.378943 11402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:01.389339 11408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:01.389386 11407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:01.389403 11410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:01.389573 11402 server_base.cc:1034] running on GCE node
I20260620 20:09:01.390259 11402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:01.391129 11402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:01.392349 11402 hybrid_clock.cc:648] HybridClock initialized: now 1781986141392321 us; error 39 us; skew 500 ppm
Jun 20 20:09:01 dist-test-slave-l8h4 krb5kdc[11027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986141, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:01.395380 11402 init.cc:377] Logged in from keytab as kudu/127.5.91.131@KRBTEST.COM (short username kudu)
I20260620 20:09:01.396924 11402 webserver.cc:460] Webserver started at http://127.5.91.131:43103/ using document root <none> and password file <none>
I20260620 20:09:01.397663 11402 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:01.397754 11402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:01.398006 11402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:01.401036 11402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "85470658b4194cec8870ffe404f8301a"
format_stamp: "Formatted at 2026-06-20 20:09:01 on dist-test-slave-l8h4"
server_key: "5cd65aaeda71aebf4abb63d7e28f6a0a"
server_key_iv: "3c12f28fff17d6244d7790aea2a8c033"
server_key_version: "encryptionkey@0"
I20260620 20:09:01.402011 11402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "85470658b4194cec8870ffe404f8301a"
format_stamp: "Formatted at 2026-06-20 20:09:01 on dist-test-slave-l8h4"
server_key: "5cd65aaeda71aebf4abb63d7e28f6a0a"
server_key_iv: "3c12f28fff17d6244d7790aea2a8c033"
server_key_version: "encryptionkey@0"
I20260620 20:09:01.407131 11402 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260620 20:09:01.410523 11417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:01.412107 11402 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 20:09:01.412247 11402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "85470658b4194cec8870ffe404f8301a"
format_stamp: "Formatted at 2026-06-20 20:09:01 on dist-test-slave-l8h4"
server_key: "5cd65aaeda71aebf4abb63d7e28f6a0a"
server_key_iv: "3c12f28fff17d6244d7790aea2a8c033"
server_key_version: "encryptionkey@0"
I20260620 20:09:01.412376 11402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:01.429939 11402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:01.434443 11402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:01.434743 11402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:01.435407 11402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:01.436695 11402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:01.436801 11402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:01.436878 11402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:01.436905 11402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:01.446043 11402 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:35839
I20260620 20:09:01.446101 11530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:35839 every 8 connection(s)
I20260620 20:09:01.447016 11402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:09:01.453658 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 11402
I20260620 20:09:01.454041 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:09:01.454420 5486 external_mini_cluster.cc:1428] Setting key 76fc7084f05b8495609149fdc8a54020
Jun 20 20:09:01 dist-test-slave-l8h4 krb5kdc[11027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986141, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:09:01.466797 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:01.448947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:42437 (local address 127.5.91.190:34995)
0620 20:09:01.449240 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:01.449245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:01.450410 (+ 1165us) server_negotiation.cc:407] Connection header received
0620 20:09:01.453008 (+ 2598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:01.453013 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:01.453075 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:01.453154 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:01.455568 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:01.456774 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:01.458347 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:01.458867 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:01.459053 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:01.462361 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:01.462366 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:01.462369 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:01.462416 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:01.464171 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:01.464925 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:01.464928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:01.464929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:01.465024 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:01.465420 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:01.465430 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:01.465431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:01.465647 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:01.465834 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:01.466487 (+ 653us) server_negotiation.cc:299] Negotiation successful
0620 20:09:01.466666 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260620 20:09:01.467598 11533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:01.449332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:34995 (local address 127.5.91.131:42437)
0620 20:09:01.450181 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:01.450237 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:01.452656 (+ 2419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:01.453425 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:01.453436 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:01.454278 (+ 842us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:01.455053 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:01.455071 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:01.456969 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:01.456972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:01.458206 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:01.458219 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:01.459108 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:01.459112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:01.459283 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:01.460124 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:01.460141 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:01.461949 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:01.464420 (+ 2471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:01.464426 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:01.464444 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:01.464771 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:01.465130 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:01.465132 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:01.465134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:01.465289 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:01.465830 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:01.465836 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:01.466117 (+ 281us) client_negotiation.cc:769] Sending connection context
0620 20:09:01.466351 (+ 234us) client_negotiation.cc:240] Negotiation successful
0620 20:09:01.466646 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":336,"threads_started":1}
I20260620 20:09:01.469030 11531 heartbeater.cc:347] Connected to a master server at 127.5.91.190:34995
I20260620 20:09:01.469551 11531 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:01.470245 11531 heartbeater.cc:511] Master 127.5.91.190:34995 requested a full tablet report, sending...
I20260620 20:09:01.471484 11072 ts_manager.cc:194] Registered new tserver with Master: 85470658b4194cec8870ffe404f8301a (127.5.91.131:35839)
I20260620 20:09:01.472100 11072 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:42437
I20260620 20:09:01.482901 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/test-admin.keytab.
Jun 20 20:09:01 dist-test-slave-l8h4 krb5kdc[11027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:01.505131 5486 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260620 20:09:01.508335 11538 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:01.506757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50520 (local address 127.5.91.129:37511)
0620 20:09:01.507176 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:01.507184 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:01.507214 (+ 30us) server_negotiation.cc:407] Connection header received
0620 20:09:01.507274 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:01.507280 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:01.507495 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:01.507630 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:01.508170 (+ 540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50520: BlockingRecv error: recv got EOF from 127.0.0.1:50520 (error 108)
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":110,"threads_started":1}
W20260620 20:09:01.508580 5486 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/krb5cc' of type FILE
Jun 20 20:09:01 dist-test-slave-l8h4 krb5kdc[11027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:01.509768 5486 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 20:09:01 dist-test-slave-l8h4 krb5kdc[11027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:01.511153 5486 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 20:09:01 dist-test-slave-l8h4 krb5kdc[11027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.129@KRBTEST.COM
I20260620 20:09:01.524272 11538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:01.512304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50536 (local address 127.5.91.129:37511)
0620 20:09:01.512423 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:01.512428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:01.512445 (+ 17us) server_negotiation.cc:407] Connection header received
0620 20:09:01.512669 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:01.512672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:01.512717 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:01.512784 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:01.513778 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:01.514839 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:01.516102 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:01.516689 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:01.516781 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:01.519589 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:01.519599 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:01.519615 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:01.519645 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:01.521989 (+ 2344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:01.522594 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:01.522600 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:01.522601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:01.522689 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:01.523066 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:01.523072 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:01.523073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:01.523569 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:01.523752 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:01.523984 (+ 232us) server_negotiation.cc:299] Negotiation successful
0620 20:09:01.524144 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260620 20:09:01.529256 11538 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:01.528145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50540 (local address 127.5.91.129:37511)
0620 20:09:01.528356 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:01.528360 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:01.528470 (+ 110us) server_negotiation.cc:407] Connection header received
0620 20:09:01.528601 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:01.528604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:01.528647 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:01.528711 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:01.529139 (+ 428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50540: BlockingRecv error: recv got EOF from 127.0.0.1:50540 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
W20260620 20:09:01.529435 5486 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/krb5cc' of type FILE
Jun 20 20:09:01 dist-test-slave-l8h4 krb5kdc[11027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:01.530507 5486 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 20:09:01 dist-test-slave-l8h4 krb5kdc[11027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:01.531344 5486 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 20:09:01 dist-test-slave-l8h4 krb5kdc[11027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.129@KRBTEST.COM
I20260620 20:09:01.542514 11538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:01.532393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50542 (local address 127.5.91.129:37511)
0620 20:09:01.532571 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:01.532576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:01.532750 (+ 174us) server_negotiation.cc:407] Connection header received
0620 20:09:01.532824 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:01.532827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:01.532890 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:01.532985 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:01.533545 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:01.534330 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:01.535543 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:01.535794 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:01.535937 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:01.538461 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:01.538465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:01.538467 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:01.538495 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:01.540283 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:01.541069 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:01.541072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:01.541074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:01.541155 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:01.541557 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:01.541560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:01.541561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:01.541750 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:01.541879 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:01.542245 (+ 366us) server_negotiation.cc:299] Negotiation successful
0620 20:09:01.542362 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260620 20:09:01.545418 11538 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:01.544358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50556 (local address 127.5.91.129:37511)
0620 20:09:01.544510 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:01.544514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:01.544626 (+ 112us) server_negotiation.cc:407] Connection header received
0620 20:09:01.544737 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:01.544740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:01.544785 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:01.544855 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:01.545308 (+ 453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50556: BlockingRecv error: recv got EOF from 127.0.0.1:50556 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260620 20:09:01.545526 5486 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781986111135528-5486-0/krb5kdc/krb5cc' of type FILE
Jun 20 20:09:01 dist-test-slave-l8h4 krb5kdc[11027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:01.546933 5486 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 20:09:01 dist-test-slave-l8h4 krb5kdc[11027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:01.548168 5486 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 20:09:01 dist-test-slave-l8h4 krb5kdc[11027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.129@KRBTEST.COM
I20260620 20:09:01.558255 11538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:01.549170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50572 (local address 127.5.91.129:37511)
0620 20:09:01.549319 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:01.549323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:01.549359 (+ 36us) server_negotiation.cc:407] Connection header received
0620 20:09:01.549503 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:01.549506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:01.549546 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:01.549615 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:01.550125 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:01.550707 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:01.551533 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:01.551748 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:01.551835 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:01.554477 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:01.554481 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:01.554484 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:01.554511 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:01.555981 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:01.556489 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:01.556492 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:01.556493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:01.556576 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:01.557337 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:01.557340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:01.557341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:01.557530 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:01.557663 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:01.557974 (+ 311us) server_negotiation.cc:299] Negotiation successful
0620 20:09:01.558079 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 20:09:01.559923 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 11134
I20260620 20:09:01.567206 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 11268
I20260620 20:09:01.574370 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 11402
I20260620 20:09:01.580881 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 11040
2026-06-20T20:09:01Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2424 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:09:01 dist-test-slave-l8h4 krb5kdc[11551](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:09:01 dist-test-slave-l8h4 krb5kdc[11551](info): set up 2 sockets
Jun 20 20:09:01 dist-test-slave-l8h4 krb5kdc[11551](info): commencing operation
krb5kdc: starting...
W20260620 20:09:02.647572 5486 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:09:02 dist-test-slave-l8h4 krb5kdc[11551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:09:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:09:02Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.190.keytab.
Entry for principal oryx/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.190.keytab.
I20260620 20:09:02.787623 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:41567
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.190.keytab
--principal=oryx/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:46671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:41567
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:09:02.921190 11564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:02.921542 11564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:02.921854 11564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:02.927306 11564 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:09:02.927397 11564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:02.927417 11564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:02.927434 11564 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:09:02.927448 11564 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:09:02.934840 11564 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:46671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:41567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.190.keytab
--principal=oryx/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:41567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:02.936215 11564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:02.938561 11564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:02.949746 11570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:02.949750 11572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:02.950069 11569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:02.950291 11564 server_base.cc:1034] running on GCE node
I20260620 20:09:02.951035 11564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:02.952802 11564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:02.954039 11564 hybrid_clock.cc:648] HybridClock initialized: now 1781986142954028 us; error 26 us; skew 500 ppm
Jun 20 20:09:02 dist-test-slave-l8h4 krb5kdc[11551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986142, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:02.960628 11564 init.cc:377] Logged in from keytab as oryx/127.5.91.190@KRBTEST.COM (short username oryx)
I20260620 20:09:02.961681 11564 webserver.cc:460] Webserver started at http://127.5.91.190:40357/ using document root <none> and password file <none>
I20260620 20:09:02.962285 11564 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:02.962360 11564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:02.962576 11564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:02.964705 11564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "b603c56a55054a1cb6603903b4748a16"
format_stamp: "Formatted at 2026-06-20 20:09:02 on dist-test-slave-l8h4"
server_key: "80cd1a22b467842338dc2ef2acbcd3f8"
server_key_iv: "63be396b3c3bbd3539d078de935e512a"
server_key_version: "encryptionkey@0"
I20260620 20:09:02.965440 11564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "b603c56a55054a1cb6603903b4748a16"
format_stamp: "Formatted at 2026-06-20 20:09:02 on dist-test-slave-l8h4"
server_key: "80cd1a22b467842338dc2ef2acbcd3f8"
server_key_iv: "63be396b3c3bbd3539d078de935e512a"
server_key_version: "encryptionkey@0"
I20260620 20:09:02.970126 11564 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 20:09:02.973742 11579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:02.975769 11564 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260620 20:09:02.975914 11564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "b603c56a55054a1cb6603903b4748a16"
format_stamp: "Formatted at 2026-06-20 20:09:02 on dist-test-slave-l8h4"
server_key: "80cd1a22b467842338dc2ef2acbcd3f8"
server_key_iv: "63be396b3c3bbd3539d078de935e512a"
server_key_version: "encryptionkey@0"
I20260620 20:09:02.976039 11564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:02.990095 11564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:02.993857 11564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:02.994022 11564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:03.003036 11564 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:41567
I20260620 20:09:03.003372 11631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:41567 every 8 connection(s)
I20260620 20:09:03.004715 11564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:09:03.006639 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 11564
I20260620 20:09:03.006829 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:09:03.007258 5486 external_mini_cluster.cc:1428] Setting key aae730089e4dae0912f604d88696f9d2
I20260620 20:09:03.008991 11632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:03.017473 11632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16: Bootstrap starting.
Jun 20 20:09:03 dist-test-slave-l8h4 krb5kdc[11551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM
I20260620 20:09:03.020254 11632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:03.021152 11632 log.cc:824] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16: Log is configured to *not* fsync() on all Append() calls
I20260620 20:09:03.023345 11632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16: No bootstrap required, opened a new log
I20260620 20:09:03.025907 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:03.009487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39632 (local address 127.5.91.190:41567)
0620 20:09:03.009982 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:03.010002 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:03.010051 (+ 49us) server_negotiation.cc:407] Connection header received
0620 20:09:03.011025 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:03.011049 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:03.011498 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:03.011937 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:03.012832 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:03.014783 (+ 1951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:03.016160 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:03.017000 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:03.017181 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:03.020092 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:03.020099 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:03.020117 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:03.020154 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:03.022708 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:03.023457 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:03.023465 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:03.023472 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:03.023623 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:03.023917 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:03.023920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:03.023922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:03.024335 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:03.024577 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:03.024816 (+ 239us) server_negotiation.cc:299] Negotiation successful
0620 20:09:03.025128 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":133,"threads_started":1}
I20260620 20:09:03.027034 11632 raft_consensus.cc:348] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } }
I20260620 20:09:03.027307 11632 raft_consensus.cc:374] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:03.027374 11632 raft_consensus.cc:729] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b603c56a55054a1cb6603903b4748a16, State: Initialized, Role: FOLLOWER
I20260620 20:09:03.028139 11632 consensus_queue.cc:260] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } }
I20260620 20:09:03.028492 11632 raft_consensus.cc:388] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:09:03.028600 11632 raft_consensus.cc:482] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:09:03.028741 11632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:03.030087 11632 raft_consensus.cc:504] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } }
I20260620 20:09:03.030936 11632 leader_election.cc:302] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b603c56a55054a1cb6603903b4748a16; no voters:
I20260620 20:09:03.031335 11632 leader_election.cc:288] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:09:03.031539 11637 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:09:03.031737 11637 raft_consensus.cc:686] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [term 1 LEADER]: Becoming Leader. State: Replica: b603c56a55054a1cb6603903b4748a16, State: Running, Role: LEADER
I20260620 20:09:03.032083 11637 consensus_queue.cc:237] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } }
I20260620 20:09:03.032737 11632 sys_catalog.cc:564] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:09:03.033409 11639 sys_catalog.cc:455] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b603c56a55054a1cb6603903b4748a16. Latest consensus state: current_term: 1 leader_uuid: "b603c56a55054a1cb6603903b4748a16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } }
I20260620 20:09:03.033569 11639 sys_catalog.cc:458] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:03.033746 11638 sys_catalog.cc:455] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b603c56a55054a1cb6603903b4748a16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } }
I20260620 20:09:03.033842 11638 sys_catalog.cc:458] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [sys.catalog]: This master's current role is: LEADER
W20260620 20:09:03.038077 11652 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 20:09:03.038213 11652 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 20:09:03.038287 11646 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:09:03.038988 11646 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:09:03.047528 11646 catalog_manager.cc:1347] Generated new cluster ID: 81149eaee63b41e9af14e730a116d9b8
I20260620 20:09:03.047627 11646 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:09:03.059353 11646 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:09:03.060403 11646 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:09:03.073859 11646 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16: Generated new TSK 0
I20260620 20:09:03.075348 11646 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.91.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.91.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.91.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.129.keytab.
Entry for principal oryx/127.5.91.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.129.keytab.
I20260620 20:09:03.117565 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41567
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.129.keytab
--principal=oryx/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:46671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:09:03.262036 11658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:03.262626 11658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:03.262701 11658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:03.268203 11658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:03.268435 11658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:03.268786 11658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:09:03.276237 11658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:46671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.129.keytab
--principal=oryx/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:03.277797 11658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:03.279178 11658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:03.291674 11663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:03.291626 11666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:03.291729 11664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:03.292894 11658 server_base.cc:1034] running on GCE node
I20260620 20:09:03.293478 11658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:03.294914 11658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:03.296196 11658 hybrid_clock.cc:648] HybridClock initialized: now 1781986143296164 us; error 40 us; skew 500 ppm
Jun 20 20:09:03 dist-test-slave-l8h4 krb5kdc[11551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986143, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:03.301100 11658 init.cc:377] Logged in from keytab as oryx/127.5.91.129@KRBTEST.COM (short username oryx)
I20260620 20:09:03.302994 11658 webserver.cc:460] Webserver started at http://127.5.91.129:37347/ using document root <none> and password file <none>
I20260620 20:09:03.303941 11658 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:03.304047 11658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:03.304275 11658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:03.307039 11658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "75c3d02cf5a54f0f8dbf93edc270a213"
format_stamp: "Formatted at 2026-06-20 20:09:03 on dist-test-slave-l8h4"
server_key: "96db82f19ae5f2bdd9f46f558d7455c9"
server_key_iv: "7fcc4ca3bb447e78711bf304ca2042b3"
server_key_version: "encryptionkey@0"
I20260620 20:09:03.307744 11658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "75c3d02cf5a54f0f8dbf93edc270a213"
format_stamp: "Formatted at 2026-06-20 20:09:03 on dist-test-slave-l8h4"
server_key: "96db82f19ae5f2bdd9f46f558d7455c9"
server_key_iv: "7fcc4ca3bb447e78711bf304ca2042b3"
server_key_version: "encryptionkey@0"
I20260620 20:09:03.313117 11658 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 20:09:03.317824 11673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:03.319936 11658 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260620 20:09:03.320098 11658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "75c3d02cf5a54f0f8dbf93edc270a213"
format_stamp: "Formatted at 2026-06-20 20:09:03 on dist-test-slave-l8h4"
server_key: "96db82f19ae5f2bdd9f46f558d7455c9"
server_key_iv: "7fcc4ca3bb447e78711bf304ca2042b3"
server_key_version: "encryptionkey@0"
I20260620 20:09:03.320240 11658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:03.352959 11658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:03.358264 11658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:03.358649 11658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:03.359975 11658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:03.361785 11658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:03.361871 11658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:03.361945 11658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:03.361972 11658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:03.372593 11658 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:33021
I20260620 20:09:03.372653 11786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:33021 every 8 connection(s)
I20260620 20:09:03.373853 11658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:09:03.377203 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 11658
I20260620 20:09:03.377331 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:09:03.377879 5486 external_mini_cluster.cc:1428] Setting key bcf1a8dbb0cfd897f3de457fa75e7fe3
Jun 20 20:09:03 dist-test-slave-l8h4 krb5kdc[11551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986143, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.5.91.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.91.130@KRBTEST.COM" created.
I20260620 20:09:03.394491 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:03.376346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:45547 (local address 127.5.91.190:41567)
0620 20:09:03.376497 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:03.376502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:03.378317 (+ 1815us) server_negotiation.cc:407] Connection header received
0620 20:09:03.380344 (+ 2027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:03.380348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:03.380408 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:03.380488 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:03.382171 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:03.382766 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:03.384603 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:03.384918 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:03.385030 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:03.389154 (+ 4124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:03.389159 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:03.389162 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:03.389201 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:03.390931 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:03.392050 (+ 1119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:03.392054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:03.392056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:03.392188 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:03.392960 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:03.392965 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:03.392967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:03.393234 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:03.393332 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:03.394206 (+ 874us) server_negotiation.cc:299] Negotiation successful
0620 20:09:03.394354 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260620 20:09:03.396412 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:03.377052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.5.91.129:45547)
0620 20:09:03.377945 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:03.378119 (+ 174us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:03.380054 (+ 1935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:03.380833 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:03.380844 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:03.381285 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:03.381819 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:03.381837 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:03.382909 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:03.382921 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:03.384361 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:03.384375 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:03.385160 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:03.385164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:03.385391 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:03.386245 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:03.386271 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:03.388823 (+ 2552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:03.391178 (+ 2355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:03.391187 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:03.391211 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:03.391865 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:03.392492 (+ 627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:03.392498 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:03.392500 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:03.392771 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:03.393394 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:03.393404 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:03.393807 (+ 403us) client_negotiation.cc:769] Sending connection context
0620 20:09:03.394098 (+ 291us) client_negotiation.cc:240] Negotiation successful
0620 20:09:03.394748 (+ 650us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":195,"threads_started":1}
I20260620 20:09:03.398406 11787 heartbeater.cc:347] Connected to a master server at 127.5.91.190:41567
I20260620 20:09:03.398741 11787 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:03.399441 11787 heartbeater.cc:511] Master 127.5.91.190:41567 requested a full tablet report, sending...
I20260620 20:09:03.401925 11594 ts_manager.cc:194] Registered new tserver with Master: 75c3d02cf5a54f0f8dbf93edc270a213 (127.5.91.129:33021)
I20260620 20:09:03.403904 11594 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:45547
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.91.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.130.keytab.
Entry for principal oryx/127.5.91.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.130.keytab.
I20260620 20:09:03.418684 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41567
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.130.keytab
--principal=oryx/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:46671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:09:03.553326 11792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:03.553647 11792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:03.553699 11792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:03.558707 11792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:03.558792 11792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:03.558854 11792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:09:03.564627 11792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:46671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.130.keytab
--principal=oryx/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:03.565755 11792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:03.566979 11792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:03.578316 11800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:03.578305 11797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:03.578305 11798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:03.579188 11792 server_base.cc:1034] running on GCE node
I20260620 20:09:03.579694 11792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:03.580440 11792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:03.581669 11792 hybrid_clock.cc:648] HybridClock initialized: now 1781986143581642 us; error 49 us; skew 500 ppm
Jun 20 20:09:03 dist-test-slave-l8h4 krb5kdc[11551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986143, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:03.585220 11792 init.cc:377] Logged in from keytab as oryx/127.5.91.130@KRBTEST.COM (short username oryx)
I20260620 20:09:03.586257 11792 webserver.cc:460] Webserver started at http://127.5.91.130:35095/ using document root <none> and password file <none>
I20260620 20:09:03.586856 11792 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:03.586939 11792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:03.587136 11792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:03.589329 11792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "daade419eb9f498994ac15c2bfa4911b"
format_stamp: "Formatted at 2026-06-20 20:09:03 on dist-test-slave-l8h4"
server_key: "02fc4501a12253028345b0a71b8928af"
server_key_iv: "eb8c7a6214e74a49384fddecdadadf80"
server_key_version: "encryptionkey@0"
I20260620 20:09:03.590377 11792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "daade419eb9f498994ac15c2bfa4911b"
format_stamp: "Formatted at 2026-06-20 20:09:03 on dist-test-slave-l8h4"
server_key: "02fc4501a12253028345b0a71b8928af"
server_key_iv: "eb8c7a6214e74a49384fddecdadadf80"
server_key_version: "encryptionkey@0"
I20260620 20:09:03.596113 11792 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260620 20:09:03.600636 11807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:03.602847 11792 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260620 20:09:03.603048 11792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "daade419eb9f498994ac15c2bfa4911b"
format_stamp: "Formatted at 2026-06-20 20:09:03 on dist-test-slave-l8h4"
server_key: "02fc4501a12253028345b0a71b8928af"
server_key_iv: "eb8c7a6214e74a49384fddecdadadf80"
server_key_version: "encryptionkey@0"
I20260620 20:09:03.603190 11792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:03.620839 11792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:03.625317 11792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:03.625541 11792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:03.626214 11792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:03.627323 11792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:03.627386 11792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:03.627456 11792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:03.627486 11792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:03.638931 11792 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:45725
I20260620 20:09:03.638947 11920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:45725 every 8 connection(s)
I20260620 20:09:03.639968 11792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:09:03.648614 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 11792
I20260620 20:09:03.648904 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:09:03.649360 5486 external_mini_cluster.cc:1428] Setting key 28d66f2b8b087928a96f9a8d31a30285
Jun 20 20:09:03 dist-test-slave-l8h4 krb5kdc[11551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986143, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM
I20260620 20:09:03.658571 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:03.642120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:54981 (local address 127.5.91.190:41567)
0620 20:09:03.642331 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:03.642336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:03.643497 (+ 1161us) server_negotiation.cc:407] Connection header received
0620 20:09:03.645419 (+ 1922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:03.645422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:03.645486 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:03.645755 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:03.647166 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:03.647752 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:03.649764 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:03.650270 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:03.650541 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:03.653956 (+ 3415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:03.653961 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:03.653964 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:03.654000 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:03.655557 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:03.656679 (+ 1122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:03.656683 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:03.656685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:03.656792 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:03.657191 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:03.657194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:03.657196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:03.657455 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:03.657567 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:03.658237 (+ 670us) server_negotiation.cc:299] Negotiation successful
0620 20:09:03.658386 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
WARNING: no policy specified for oryx/127.5.91.131@KRBTEST.COM; defaulting to no policy
I20260620 20:09:03.660044 11923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:03.642515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.5.91.130:54981)
0620 20:09:03.643203 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:03.643283 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:03.645108 (+ 1825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:03.645970 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:03.645979 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:03.646448 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:03.646990 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:03.647006 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:03.647904 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:03.647907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:03.649534 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:03.649549 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:03.651111 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:03.651115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:03.651378 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:03.652000 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:03.652019 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:03.653688 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:03.655743 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:03.655751 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:03.655774 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:03.656502 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:03.656923 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:03.656926 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:03.656927 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:03.657089 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:03.657608 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:03.657614 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:03.657888 (+ 274us) client_negotiation.cc:769] Sending connection context
0620 20:09:03.658123 (+ 235us) client_negotiation.cc:240] Negotiation successful
0620 20:09:03.658500 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":125,"threads_started":1}
I20260620 20:09:03.662204 11921 heartbeater.cc:347] Connected to a master server at 127.5.91.190:41567
I20260620 20:09:03.662518 11921 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:03.663072 11921 heartbeater.cc:511] Master 127.5.91.190:41567 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.91.131@KRBTEST.COM" created.
I20260620 20:09:03.664498 11594 ts_manager.cc:194] Registered new tserver with Master: daade419eb9f498994ac15c2bfa4911b (127.5.91.130:45725)
I20260620 20:09:03.665122 11594 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:54981
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.91.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.131.keytab.
Entry for principal oryx/127.5.91.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.131.keytab.
I20260620 20:09:03.683656 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41567
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.131.keytab
--principal=oryx/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:46671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:09:03.824221 11926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:03.824579 11926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:03.824644 11926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:03.830196 11926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:03.830288 11926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:03.830389 11926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:09:03.838295 11926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:46671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.131.keytab
--principal=oryx/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:03.839422 11926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:03.840438 11926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:03.852741 11932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:03.852756 11934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:03.853116 11931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:03.853180 11926 server_base.cc:1034] running on GCE node
I20260620 20:09:03.853672 11926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:03.854429 11926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:03.855609 11926 hybrid_clock.cc:648] HybridClock initialized: now 1781986143855585 us; error 40 us; skew 500 ppm
Jun 20 20:09:03 dist-test-slave-l8h4 krb5kdc[11551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986143, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:03.859511 11926 init.cc:377] Logged in from keytab as oryx/127.5.91.131@KRBTEST.COM (short username oryx)
I20260620 20:09:03.860921 11926 webserver.cc:460] Webserver started at http://127.5.91.131:35361/ using document root <none> and password file <none>
I20260620 20:09:03.861568 11926 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:03.861642 11926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:03.861841 11926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:03.865384 11926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "b728814d629c46d2ac07d56b1dff1e6e"
format_stamp: "Formatted at 2026-06-20 20:09:03 on dist-test-slave-l8h4"
server_key: "05aca4110eb6caf597dc7d733ee327ae"
server_key_iv: "06953f90d74a9b156fa6eefe3fddd1f4"
server_key_version: "encryptionkey@0"
I20260620 20:09:03.866261 11926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "b728814d629c46d2ac07d56b1dff1e6e"
format_stamp: "Formatted at 2026-06-20 20:09:03 on dist-test-slave-l8h4"
server_key: "05aca4110eb6caf597dc7d733ee327ae"
server_key_iv: "06953f90d74a9b156fa6eefe3fddd1f4"
server_key_version: "encryptionkey@0"
I20260620 20:09:03.870772 11926 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260620 20:09:03.875577 11941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:03.878291 11926 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.007s sys 0.000s
I20260620 20:09:03.878504 11926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "b728814d629c46d2ac07d56b1dff1e6e"
format_stamp: "Formatted at 2026-06-20 20:09:03 on dist-test-slave-l8h4"
server_key: "05aca4110eb6caf597dc7d733ee327ae"
server_key_iv: "06953f90d74a9b156fa6eefe3fddd1f4"
server_key_version: "encryptionkey@0"
I20260620 20:09:03.878631 11926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:03.913486 11926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:03.918717 11926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:03.919262 11926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:03.920204 11926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:03.921617 11926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:03.921681 11926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:03.921738 11926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:03.921772 11926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:03.934584 11926 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:39363
I20260620 20:09:03.934672 12054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:39363 every 8 connection(s)
I20260620 20:09:03.935844 11926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:09:03.943032 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 11926
I20260620 20:09:03.943269 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:09:03.943894 5486 external_mini_cluster.cc:1428] Setting key 2f868e3b249ce0dfbdf6575914c90d84
Jun 20 20:09:03 dist-test-slave-l8h4 krb5kdc[11551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986143, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM
I20260620 20:09:03.956981 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:03.938480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:49593 (local address 127.5.91.190:41567)
0620 20:09:03.938873 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:03.938879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:03.941391 (+ 2512us) server_negotiation.cc:407] Connection header received
0620 20:09:03.942788 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:03.942791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:03.942868 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:03.942943 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:03.944440 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:03.945137 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:03.947277 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:03.947551 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:03.947666 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:03.951069 (+ 3403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:03.951075 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:03.951077 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:03.951122 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:03.953492 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:03.954382 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:03.954386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:03.954388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:03.954505 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:03.955369 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:03.955372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:03.955373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:03.955620 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:03.955725 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:03.956700 (+ 975us) server_negotiation.cc:299] Negotiation successful
0620 20:09:03.956841 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246}
I20260620 20:09:03.958022 12057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:03.939769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.5.91.131:49593)
0620 20:09:03.940982 (+ 1213us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:03.941051 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:03.942501 (+ 1450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:03.943158 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:03.943169 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:03.943634 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:03.944208 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:03.944223 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:03.945448 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:03.945453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:03.947122 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:03.947138 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:03.947748 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:03.947752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:03.947943 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:03.948619 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:03.948641 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:03.950623 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:03.953780 (+ 3157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:03.953788 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:03.953808 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:03.954183 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:03.954767 (+ 584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:03.954771 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:03.954773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:03.955059 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:03.955787 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:03.955796 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:03.956233 (+ 437us) client_negotiation.cc:769] Sending connection context
0620 20:09:03.956585 (+ 352us) client_negotiation.cc:240] Negotiation successful
0620 20:09:03.956948 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":954,"thread_start_us":743,"threads_started":1}
I20260620 20:09:03.959573 12055 heartbeater.cc:347] Connected to a master server at 127.5.91.190:41567
I20260620 20:09:03.959829 12055 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:03.960399 12055 heartbeater.cc:511] Master 127.5.91.190:41567 requested a full tablet report, sending...
I20260620 20:09:03.961783 11594 ts_manager.cc:194] Registered new tserver with Master: b728814d629c46d2ac07d56b1dff1e6e (127.5.91.131:39363)
I20260620 20:09:03.962579 11594 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:49593
I20260620 20:09:03.973330 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.91.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.91.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.91.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.189.keytab.
Entry for principal oryx/127.5.91.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.189.keytab.
I20260620 20:09:04.011731 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.189:34727
--webserver_interface=127.5.91.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.189.keytab
--principal=oryx/127.5.91.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:46671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:41567,127.5.91.189:34727
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:09:04.147923 12060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:04.148226 12060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:04.148286 12060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:04.153167 12060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:09:04.153306 12060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:04.153347 12060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:04.153370 12060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:09:04.153390 12060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:09:04.161125 12060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:46671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:41567,127.5.91.189:34727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.189.keytab
--principal=oryx/127.5.91.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.189:34727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.91.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
W20260620 20:09:04.161401 12060 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.5.91.190:41567,127.5.91.189:34727'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260620 20:09:04.162694 12060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:04.163993 12060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:04.174399 12065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:04.174401 12068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:04.174420 12066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:04.174670 12060 server_base.cc:1034] running on GCE node
I20260620 20:09:04.175391 12060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:04.176635 12060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:04.177979 12060 hybrid_clock.cc:648] HybridClock initialized: now 1781986144177913 us; error 47 us; skew 500 ppm
Jun 20 20:09:04 dist-test-slave-l8h4 krb5kdc[11551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986144, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:04.182391 12060 init.cc:377] Logged in from keytab as oryx/127.5.91.189@KRBTEST.COM (short username oryx)
I20260620 20:09:04.183405 12060 webserver.cc:460] Webserver started at http://127.5.91.189:45143/ using document root <none> and password file <none>
I20260620 20:09:04.184031 12060 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:04.184114 12060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:04.184305 12060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:04.186502 12060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/data/instance:
uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada"
format_stamp: "Formatted at 2026-06-20 20:09:04 on dist-test-slave-l8h4"
server_key: "81f4e595e6c8b3374d945993361b6814"
server_key_iv: "dbcc5c94f59676012a277528191eedca"
server_key_version: "encryptionkey@0"
I20260620 20:09:04.187085 12060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/wal/instance:
uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada"
format_stamp: "Formatted at 2026-06-20 20:09:04 on dist-test-slave-l8h4"
server_key: "81f4e595e6c8b3374d945993361b6814"
server_key_iv: "dbcc5c94f59676012a277528191eedca"
server_key_version: "encryptionkey@0"
I20260620 20:09:04.191368 12060 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260620 20:09:04.193969 12075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:04.195670 12060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:09:04.195809 12060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/wal
uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada"
format_stamp: "Formatted at 2026-06-20 20:09:04 on dist-test-slave-l8h4"
server_key: "81f4e595e6c8b3374d945993361b6814"
server_key_iv: "dbcc5c94f59676012a277528191eedca"
server_key_version: "encryptionkey@0"
I20260620 20:09:04.195963 12060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:04.209280 12060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:04.213003 12060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:04.213182 12060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:04.220785 12060 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.189:34727
I20260620 20:09:04.221357 12127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.189:34727 every 8 connection(s)
I20260620 20:09:04.221904 12060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/data/info.pb
I20260620 20:09:04.225148 12128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:04.229404 12128 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } has no permanent_uuid. Determining permanent_uuid...
I20260620 20:09:04.230618 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 12060
Jun 20 20:09:04 dist-test-slave-l8h4 krb5kdc[11551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986144, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.189@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM
Jun 20 20:09:04 dist-test-slave-l8h4 krb5kdc[11551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.91.189@KRBTEST.COM
I20260620 20:09:04.247387 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.231557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39636 (local address 127.5.91.190:41567)
0620 20:09:04.232454 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:04.232459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:04.232813 (+ 354us) server_negotiation.cc:407] Connection header received
0620 20:09:04.234262 (+ 1449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:04.234266 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:04.234333 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:04.234472 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:04.235769 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.236713 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.238461 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.238883 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.239016 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.242083 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:04.242086 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:04.242088 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:04.242115 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:04.244051 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.244624 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.244628 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.244631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.244722 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.245656 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.245659 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.245660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.245966 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:04.246076 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:04.247066 (+ 990us) server_negotiation.cc:299] Negotiation successful
0620 20:09:04.247214 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":809}
I20260620 20:09:04.248263 12132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.232070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.0.0.1:39636)
0620 20:09:04.232657 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:04.232708 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:04.233868 (+ 1160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:04.234547 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:04.234557 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:04.235166 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:04.235557 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.235580 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.236960 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.236965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.238279 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.238291 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.239084 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.239087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.239228 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.240062 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:04.240082 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:04.241912 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:04.244186 (+ 2274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.244194 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.244204 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.244515 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.245371 (+ 856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.245374 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.245376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.245540 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.246457 (+ 917us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:04.246485 (+ 28us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:04.246744 (+ 259us) client_negotiation.cc:769] Sending connection context
0620 20:09:04.246940 (+ 196us) client_negotiation.cc:240] Negotiation successful
0620 20:09:04.247860 (+ 920us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"spinlock_wait_cycles":2944,"thread_start_us":149,"threads_started":1}
I20260620 20:09:04.248273 12131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.231990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47196 (local address 127.5.91.189:34727)
0620 20:09:04.232429 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:04.232447 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:04.232487 (+ 40us) server_negotiation.cc:407] Connection header received
0620 20:09:04.233849 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:04.233883 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:04.234399 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:04.234793 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:04.235679 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.237192 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.238454 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.239058 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.239209 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.242690 (+ 3481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:04.242701 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:04.242736 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:04.242781 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:04.244719 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.245389 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.245395 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.245397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.245516 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.245805 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.245808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.245810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.246330 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:04.246547 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:04.247071 (+ 524us) server_negotiation.cc:299] Negotiation successful
0620 20:09:04.247334 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":136,"threads_started":1}
I20260620 20:09:04.249708 12128 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 20:09:04 dist-test-slave-l8h4 krb5kdc[11551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986144, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.189@KRBTEST.COM for oryx/127.5.91.189@KRBTEST.COM
I20260620 20:09:04.259671 12132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.250229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:34727 (local address 127.0.0.1:47210)
0620 20:09:04.250346 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:04.250360 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:04.250437 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:04.250619 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:04.250622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:04.250857 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:04.250971 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.250977 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.251661 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.251664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.252176 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.252183 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.252653 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.252657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.252940 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.253484 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:04.253502 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:04.255876 (+ 2374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:04.257694 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.257697 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.257699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.258029 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.258647 (+ 618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.258649 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.258651 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.258735 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.259163 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:04.259165 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:04.259253 (+ 88us) client_negotiation.cc:769] Sending connection context
0620 20:09:04.259348 (+ 95us) client_negotiation.cc:240] Negotiation successful
0620 20:09:04.259539 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260620 20:09:04.259843 12131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.250351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47210 (local address 127.5.91.189:34727)
0620 20:09:04.250432 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:04.250436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:04.250448 (+ 12us) server_negotiation.cc:407] Connection header received
0620 20:09:04.250504 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:04.250507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:04.250541 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:04.250599 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:04.251063 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.251573 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.252276 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.252558 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.252645 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.256047 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:04.256051 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:04.256053 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:04.256077 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:04.257560 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.258264 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.258270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.258273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.258472 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.258873 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.258876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.258877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.259067 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:04.259156 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:04.259412 (+ 256us) server_negotiation.cc:299] Negotiation successful
0620 20:09:04.259681 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":156,"server-negotiator.queue_time_us":36}
I20260620 20:09:04.265245 12128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada: Bootstrap starting.
I20260620 20:09:04.267920 12128 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:04.268828 12128 log.cc:824] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada: Log is configured to *not* fsync() on all Append() calls
I20260620 20:09:04.272059 12128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada: No bootstrap required, opened a new log
I20260620 20:09:04.275300 12128 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } }
I20260620 20:09:04.275542 12128 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:04.275575 12128 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e00b0caef7c4ba7afd5f5a6ced4fada, State: Initialized, Role: FOLLOWER
I20260620 20:09:04.276034 12128 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } }
I20260620 20:09:04.276808 12134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } } }
I20260620 20:09:04.276964 12134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:09:04.277308 12128 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [sys.catalog]: configured and running, proceeding with master startup.
W20260620 20:09:04.282974 12145 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 20:09:04.283131 12145 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 20:09:04.286285 12060 master_runner.cc:428] Detected that this master 0e00b0caef7c4ba7afd5f5a6ced4fada is joining an existing cluster
I20260620 20:09:04.286369 12060 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.91.189:34727
I20260620 20:09:04.298319 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.289190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39642 (local address 127.5.91.190:41567)
0620 20:09:04.289317 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:04.289322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:04.289337 (+ 15us) server_negotiation.cc:407] Connection header received
0620 20:09:04.289385 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:04.289388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:04.289439 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:04.289531 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:04.290193 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.290792 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.291988 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.292209 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.292365 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.294082 (+ 1717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:04.294089 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:04.294092 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:04.294194 (+ 102us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:04.296537 (+ 2343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.296989 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.296992 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.296993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.297070 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.297308 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.297311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.297313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.297556 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:04.297646 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:04.298014 (+ 368us) server_negotiation.cc:299] Negotiation successful
0620 20:09:04.298129 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 20:09:04.298331 12151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.288702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.0.0.1:39642)
0620 20:09:04.289019 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:04.289034 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:04.289153 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:04.289591 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:04.289594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:04.289788 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:04.290026 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.290034 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.290941 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.290944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.291853 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.291863 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.292397 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.292401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.292487 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.293407 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:04.293420 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:04.293831 (+ 411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:04.296660 (+ 2829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.296663 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.296666 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.296891 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.297149 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.297152 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.297154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.297228 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.297645 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:04.297648 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:04.297799 (+ 151us) client_negotiation.cc:769] Sending connection context
0620 20:09:04.297943 (+ 144us) client_negotiation.cc:240] Negotiation successful
0620 20:09:04.298106 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":124,"threads_started":1}
I20260620 20:09:04.300547 12152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.288716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:34727 (local address 127.0.0.1:47218)
0620 20:09:04.289772 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:04.289786 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:04.289883 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:04.290127 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:04.290130 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:04.290369 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:04.290509 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.290516 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.291402 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.291406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.292299 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.292309 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.293062 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.293072 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.293173 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.294968 (+ 1795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:04.294985 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:04.295480 (+ 495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:04.298919 (+ 3439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.298922 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.298924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.299149 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.299395 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.299398 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.299400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.299467 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.299955 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:04.299958 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:04.300057 (+ 99us) client_negotiation.cc:769] Sending connection context
0620 20:09:04.300169 (+ 112us) client_negotiation.cc:240] Negotiation successful
0620 20:09:04.300329 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":989,"thread_start_us":105,"threads_started":1}
I20260620 20:09:04.300554 12131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.289142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47218 (local address 127.5.91.189:34727)
0620 20:09:04.289236 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:04.289240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:04.289884 (+ 644us) server_negotiation.cc:407] Connection header received
0620 20:09:04.289967 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:04.289970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:04.290018 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:04.290097 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:04.290623 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.291276 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.292420 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.292735 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.292803 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.295677 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:04.295681 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:04.295683 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:04.295709 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:04.298820 (+ 3111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.299232 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.299243 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.299245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.299321 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.299539 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.299542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.299544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.299727 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:04.299815 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:04.300255 (+ 440us) server_negotiation.cc:299] Negotiation successful
0620 20:09:04.300397 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":39}
WARNING: no policy specified for oryx/127.5.91.188@KRBTEST.COM; defaulting to no policy
Jun 20 20:09:04 dist-test-slave-l8h4 krb5kdc[11551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986142, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.190@KRBTEST.COM for oryx/127.5.91.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.91.188@KRBTEST.COM" created.
I20260620 20:09:04.321035 12131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.307420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47222 (local address 127.5.91.189:34727)
0620 20:09:04.307559 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:04.307564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:04.307578 (+ 14us) server_negotiation.cc:407] Connection header received
0620 20:09:04.307700 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:04.307703 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:04.307756 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:04.307847 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:04.308908 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.309394 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.310521 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.310745 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.310900 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.315497 (+ 4597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:04.315503 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:04.315506 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:04.315546 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:04.317877 (+ 2331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.318924 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.318930 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.318932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.319133 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.319612 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.319615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.319617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.319911 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:04.320008 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:04.320701 (+ 693us) server_negotiation.cc:299] Negotiation successful
0620 20:09:04.320840 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 20:09:04.321161 12154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.306353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:34727 (local address 127.0.0.1:47222)
0620 20:09:04.306856 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:04.306924 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:04.307430 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:04.307925 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:04.307932 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:04.308427 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:04.308720 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.308737 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.309510 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.309512 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.310385 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.310396 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.310864 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.310868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.310999 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.311991 (+ 992us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:04.312010 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:04.315166 (+ 3156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:04.318091 (+ 2925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.318112 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.318121 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.318553 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.319333 (+ 780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.319338 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.319340 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.319485 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.320103 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:04.320113 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:04.320302 (+ 189us) client_negotiation.cc:769] Sending connection context
0620 20:09:04.320532 (+ 230us) client_negotiation.cc:240] Negotiation successful
0620 20:09:04.320905 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":143,"threads_started":1}
I20260620 20:09:04.322898 11594 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.91.189:34727: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada"
member_type: NON_VOTER
last_known_addr {
host: "127.5.91.189"
port: 34727
}
attrs {
promote: true
}
}
dest_uuid: "b603c56a55054a1cb6603903b4748a16"
cas_config_opid_index: -1
I20260620 20:09:04.323598 11594 consensus_queue.cc:237] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: NON_VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: true } }
I20260620 20:09:04.326574 12157 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0e00b0caef7c4ba7afd5f5a6ced4fada (127.5.91.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: NON_VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: true } } }
I20260620 20:09:04.327483 12102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:04.328542 12157 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.91.189:34727
I20260620 20:09:04.328927 12102 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [term 1 FOLLOWER]: Refusing update from remote peer b603c56a55054a1cb6603903b4748a16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260620 20:09:04.329049 12060 master.cc:537] Master@127.5.91.189:34727 shutting down...
I20260620 20:09:04.328917 12157 sys_catalog.cc:455] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b603c56a55054a1cb6603903b4748a16" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: NON_VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: true } } }
I20260620 20:09:04.329460 12157 sys_catalog.cc:458] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:04.330008 12158 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: NON_VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 20:09:04.332221 11580 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260620 20:09:04.333385 11580 consensus_peers.cc:487] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 -> Peer 0e00b0caef7c4ba7afd5f5a6ced4fada (127.5.91.189:34727): Couldn't send request to peer 0e00b0caef7c4ba7afd5f5a6ced4fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260620 20:09:04.334633 12060 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:09:04.334923 12060 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:09:04.335026 12060 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada: stopping tablet replica
I20260620 20:09:04.344870 12060 master.cc:552] Master@127.5.91.189:34727 shutdown complete.
I20260620 20:09:04.344976 12060 master_runner.cc:305] Clearing existing system tablet
I20260620 20:09:04.345213 12060 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:04.345293 12060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:04.347138 12060 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 20:09:04.348233 12162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:04.348546 12060 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:04.348673 12060 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 20:09:04.348707 12060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/wal
uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada"
format_stamp: "Formatted at 2026-06-20 20:09:04 on dist-test-slave-l8h4"
server_key: "81f4e595e6c8b3374d945993361b6814"
server_key_iv: "dbcc5c94f59676012a277528191eedca"
server_key_version: "encryptionkey@0"
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.91.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.188.keytab.
Entry for principal oryx/127.5.91.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.188.keytab.
I20260620 20:09:04.351336 12060 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:09:04.352172 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.188:41005
--webserver_interface=127.5.91.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.188.keytab
--principal=oryx/127.5.91.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:46671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:41567,127.5.91.189:34727,127.5.91.188:41005
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 20:09:04.353832 12060 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260620 20:09:04.353930 12060 log.cc:1196] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260620 20:09:04.354247 12060 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada: Deleting consensus metadata
I20260620 20:09:04.354756 12060 master_runner.cc:315] Copying system tablet from 127.5.91.190:41567
I20260620 20:09:04.355980 12060 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada: tablet copy: Beginning tablet copy session from remote peer at address 127.5.91.190:41567
I20260620 20:09:04.366835 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.356820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39656 (local address 127.5.91.190:41567)
0620 20:09:04.357111 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:04.357117 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:04.357263 (+ 146us) server_negotiation.cc:407] Connection header received
0620 20:09:04.357418 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:04.357421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:04.357479 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:04.357588 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:04.358268 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.359291 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.360517 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.360893 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.361090 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.363064 (+ 1974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:04.363068 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:04.363071 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:04.363100 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:04.364946 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.365479 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.365483 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.365485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.365579 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.365842 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.365845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.365847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.366088 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:04.366251 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:04.366530 (+ 279us) server_negotiation.cc:299] Negotiation successful
0620 20:09:04.366659 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260620 20:09:04.366845 12169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.356549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.0.0.1:39656)
0620 20:09:04.357144 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:04.357168 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:04.357315 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:04.357691 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:04.357694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:04.357916 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:04.358131 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.358166 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.359431 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.359435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.360373 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.360385 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.361203 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.361210 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.361371 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.362298 (+ 927us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:04.362342 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:04.362858 (+ 516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:04.365093 (+ 2235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.365098 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.365100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.365372 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.365665 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.365668 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.365670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.365756 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.366239 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:04.366242 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:04.366357 (+ 115us) client_negotiation.cc:769] Sending connection context
0620 20:09:04.366463 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 20:09:04.366624 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":99,"threads_started":1}
I20260620 20:09:04.367880 11616 tablet_copy_service.cc:140] P b603c56a55054a1cb6603903b4748a16: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0e00b0caef7c4ba7afd5f5a6ced4fada ({username='oryx', principal='oryx/127.5.91.189@KRBTEST.COM'} at 127.0.0.1:39656)
I20260620 20:09:04.368078 11616 tablet_copy_service.cc:161] P b603c56a55054a1cb6603903b4748a16: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0e00b0caef7c4ba7afd5f5a6ced4fada at {username='oryx', principal='oryx/127.5.91.189@KRBTEST.COM'} at 127.0.0.1:39656: session id = 0e00b0caef7c4ba7afd5f5a6ced4fada-00000000000000000000000000000000
I20260620 20:09:04.370060 11616 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 20:09:04.372017 12060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260620 20:09:04.374687 12060 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada: tablet copy: Starting download of 0 data blocks...
I20260620 20:09:04.374889 12060 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada: tablet copy: Starting download of 1 WAL segments...
I20260620 20:09:04.379802 12060 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 20:09:04.383405 11616 tablet_copy_service.cc:342] P b603c56a55054a1cb6603903b4748a16: Request end of tablet copy session 0e00b0caef7c4ba7afd5f5a6ced4fada-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.91.189@KRBTEST.COM'} at 127.0.0.1:39656
I20260620 20:09:04.383673 11616 tablet_copy_service.cc:433] P b603c56a55054a1cb6603903b4748a16: ending tablet copy session 0e00b0caef7c4ba7afd5f5a6ced4fada-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0e00b0caef7c4ba7afd5f5a6ced4fada
I20260620 20:09:04.393992 12060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:04.396214 12170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:04.396611 12173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:04.396629 12171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:04.397040 12060 server_base.cc:1034] running on GCE node
I20260620 20:09:04.397195 12060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:04.397500 12060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:04.398679 12060 hybrid_clock.cc:648] HybridClock initialized: now 1781986144398667 us; error 30 us; skew 500 ppm
Jun 20 20:09:04 dist-test-slave-l8h4 krb5kdc[11551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986144, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:04.401170 12060 init.cc:377] Logged in from keytab as oryx/127.5.91.189@KRBTEST.COM (short username oryx)
I20260620 20:09:04.401733 12060 webserver.cc:460] Webserver started at http://127.5.91.189:39323/ using document root <none> and password file <none>
I20260620 20:09:04.401919 12060 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:04.401960 12060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:04.403286 12060 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 20:09:04.404381 12180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:04.404663 12060 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 20:09:04.404742 12060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/wal
uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada"
format_stamp: "Formatted at 2026-06-20 20:09:04 on dist-test-slave-l8h4"
server_key: "81f4e595e6c8b3374d945993361b6814"
server_key_iv: "dbcc5c94f59676012a277528191eedca"
server_key_version: "encryptionkey@0"
I20260620 20:09:04.404819 12060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:04.408113 11787 heartbeater.cc:503] Master 127.5.91.190:41567 was elected leader, sending a full tablet report...
I20260620 20:09:04.434046 12060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:04.436578 12060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:04.441347 12060 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.189:34727
I20260620 20:09:04.441380 12232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.189:34727 every 8 connection(s)
I20260620 20:09:04.442487 12060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-1/data/info.pb
I20260620 20:09:04.444947 12233 sys_catalog.cc:263] Verifying existing consensus state
I20260620 20:09:04.446185 12233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada: Bootstrap starting.
I20260620 20:09:04.457180 12233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 20:09:04.457535 12233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada: Bootstrap complete.
I20260620 20:09:04.458117 12233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: NON_VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: true } }
I20260620 20:09:04.458277 12233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0e00b0caef7c4ba7afd5f5a6ced4fada, State: Initialized, Role: LEARNER
I20260620 20:09:04.458380 12233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: NON_VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: true } }
I20260620 20:09:04.459136 12233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:09:04.459017 12236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: NON_VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: true } } }
I20260620 20:09:04.459337 12236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [sys.catalog]: This master's current role is: LEARNER
I20260620 20:09:04.462406 12247 catalog_manager.cc:1259] Loaded cluster ID: 81149eaee63b41e9af14e730a116d9b8
I20260620 20:09:04.462527 12247 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada: loading cluster ID for follower catalog manager: success
I20260620 20:09:04.464816 12247 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada: acquiring CA information for follower catalog manager: success
I20260620 20:09:04.466341 12247 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260620 20:09:04.508728 12163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:04.509080 12163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:04.509227 12163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:04.514052 12163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:09:04.514176 12163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:04.514207 12163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:04.514230 12163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:09:04.514249 12163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:09:04.520046 12163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:46671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:41567,127.5.91.189:34727,127.5.91.188:41005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.188.keytab
--principal=oryx/127.5.91.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.188:41005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.91.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:04.521394 12163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:04.522635 12163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:04.532750 12253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:04.532728 12255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:04.532728 12252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:04.533725 12163 server_base.cc:1034] running on GCE node
I20260620 20:09:04.534780 12163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:04.536144 12163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:04.537360 12163 hybrid_clock.cc:648] HybridClock initialized: now 1781986144537349 us; error 49 us; skew 500 ppm
Jun 20 20:09:04 dist-test-slave-l8h4 krb5kdc[11551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986144, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:04.541096 12163 init.cc:377] Logged in from keytab as oryx/127.5.91.188@KRBTEST.COM (short username oryx)
I20260620 20:09:04.542325 12163 webserver.cc:460] Webserver started at http://127.5.91.188:38171/ using document root <none> and password file <none>
I20260620 20:09:04.542925 12163 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:04.542996 12163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:04.543166 12163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:04.545337 12163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/data/instance:
uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170"
format_stamp: "Formatted at 2026-06-20 20:09:04 on dist-test-slave-l8h4"
server_key: "4713c1129c355303a9ef4f356252473f"
server_key_iv: "2de673447a34e9cca9f79450253f7c0c"
server_key_version: "encryptionkey@0"
I20260620 20:09:04.545919 12163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/wal/instance:
uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170"
format_stamp: "Formatted at 2026-06-20 20:09:04 on dist-test-slave-l8h4"
server_key: "4713c1129c355303a9ef4f356252473f"
server_key_iv: "2de673447a34e9cca9f79450253f7c0c"
server_key_version: "encryptionkey@0"
I20260620 20:09:04.550293 12163 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 20:09:04.553203 12262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:04.555014 12163 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:09:04.555155 12163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/wal
uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170"
format_stamp: "Formatted at 2026-06-20 20:09:04 on dist-test-slave-l8h4"
server_key: "4713c1129c355303a9ef4f356252473f"
server_key_iv: "2de673447a34e9cca9f79450253f7c0c"
server_key_version: "encryptionkey@0"
I20260620 20:09:04.555265 12163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:04.571074 12163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:04.574800 12163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:04.574977 12163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:04.582229 12163 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.188:41005
I20260620 20:09:04.582515 12314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.188:41005 every 8 connection(s)
I20260620 20:09:04.583336 12163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/data/info.pb
I20260620 20:09:04.586398 12315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:04.590296 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 12163
I20260620 20:09:04.590590 12315 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 20:09:04 dist-test-slave-l8h4 krb5kdc[11551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986144, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.188@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM
Jun 20 20:09:04 dist-test-slave-l8h4 krb5kdc[11551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.91.188@KRBTEST.COM
I20260620 20:09:04.603492 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.592838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39664 (local address 127.5.91.190:41567)
0620 20:09:04.593050 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:04.593054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:04.593214 (+ 160us) server_negotiation.cc:407] Connection header received
0620 20:09:04.593906 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:04.593909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:04.593961 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:04.594028 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:04.595250 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.595802 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.596821 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.597047 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.597132 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.599796 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:04.599801 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:04.599804 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:04.599834 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:04.601310 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.601893 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.601896 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.601898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.602008 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.602362 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.602366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.602367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.602586 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:04.602671 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:04.603186 (+ 515us) server_negotiation.cc:299] Negotiation successful
0620 20:09:04.603320 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260620 20:09:04.604503 12317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.592213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41560 (local address 127.5.91.188:41005)
0620 20:09:04.592823 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:04.592842 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:04.592884 (+ 42us) server_negotiation.cc:407] Connection header received
0620 20:09:04.593653 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:04.593672 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:04.593875 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:04.594032 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:04.594902 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.595847 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.596647 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.597011 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.597154 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.600180 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:04.600186 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:04.600199 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:04.600224 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:04.601892 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.602398 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.602404 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.602406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.602496 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.602752 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.602755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.602757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.603020 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:04.603214 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:04.603413 (+ 199us) server_negotiation.cc:299] Negotiation successful
0620 20:09:04.603780 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":279,"threads_started":1}
I20260620 20:09:04.604501 12318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.592515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.0.0.1:39664)
0620 20:09:04.593064 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:04.593117 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:04.593682 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:04.594094 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:04.594099 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:04.594845 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:04.595098 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.595110 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.595938 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.595940 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.596702 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.596711 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.597188 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.597192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.597305 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.597868 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:04.597884 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:04.599579 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:04.601443 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.601451 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.601460 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.601796 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.602087 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.602089 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.602091 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.602279 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.602679 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:04.602686 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:04.602960 (+ 274us) client_negotiation.cc:769] Sending connection context
0620 20:09:04.603115 (+ 155us) client_negotiation.cc:240] Negotiation successful
0620 20:09:04.603444 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":148,"threads_started":1}
I20260620 20:09:04.606741 12315 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 20:09:04 dist-test-slave-l8h4 krb5kdc[11551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986144, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.188@KRBTEST.COM for oryx/127.5.91.189@KRBTEST.COM
I20260620 20:09:04.618047 12318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.607345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:34727 (local address 127.0.0.1:47232)
0620 20:09:04.607497 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:04.607511 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:04.607578 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:04.608007 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:04.608011 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:04.608185 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:04.608368 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.608375 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.609697 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.609702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.610439 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.610455 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.611426 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.611430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.611542 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.612190 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:04.612204 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:04.613693 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:04.616144 (+ 2451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.616148 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.616150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.616409 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.616937 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.616941 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.616943 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.617039 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.617546 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:04.617549 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:04.617651 (+ 102us) client_negotiation.cc:769] Sending connection context
0620 20:09:04.617748 (+ 97us) client_negotiation.cc:240] Negotiation successful
0620 20:09:04.617909 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260620 20:09:04.618271 12319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.607499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47232 (local address 127.5.91.189:34727)
0620 20:09:04.607745 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:04.607748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:04.607761 (+ 13us) server_negotiation.cc:407] Connection header received
0620 20:09:04.607800 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:04.607802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:04.607840 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:04.607929 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:04.608626 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.609471 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.610701 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.611197 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.611351 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.613992 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:04.613996 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:04.613998 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:04.614036 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:04.615982 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.616660 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.616665 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.616666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.616807 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.617171 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.617174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.617176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.617424 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:04.617518 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:04.617937 (+ 419us) server_negotiation.cc:299] Negotiation successful
0620 20:09:04.618065 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":96,"threads_started":1}
I20260620 20:09:04.619096 12315 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 20:09:04 dist-test-slave-l8h4 krb5kdc[11551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986144, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.188@KRBTEST.COM for oryx/127.5.91.188@KRBTEST.COM
I20260620 20:09:04.628700 12318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.619607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:41005 (local address 127.0.0.1:41574)
0620 20:09:04.619749 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:04.619763 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:04.619841 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:04.620078 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:04.620081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:04.620238 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:04.620305 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.620311 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.621064 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.621068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.621554 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.621561 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.621950 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.621953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.622076 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.622579 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:04.622590 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:04.624305 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:04.626538 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.626543 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.626546 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.626910 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.627337 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.627341 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.627344 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.627493 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.628161 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:04.628165 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:04.628294 (+ 129us) client_negotiation.cc:769] Sending connection context
0620 20:09:04.628418 (+ 124us) client_negotiation.cc:240] Negotiation successful
0620 20:09:04.628559 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":29}
I20260620 20:09:04.628925 12317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.619718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41574 (local address 127.5.91.188:41005)
0620 20:09:04.619814 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:04.619817 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:04.619828 (+ 11us) server_negotiation.cc:407] Connection header received
0620 20:09:04.619957 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:04.619959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:04.619996 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:04.620056 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:04.620447 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.620969 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.621649 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.621859 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.621952 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.624528 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:04.624532 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:04.624535 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:04.624560 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:04.626344 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.627064 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.627067 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.627069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.627157 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.627751 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.627754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.627755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.628018 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:04.628120 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:04.628519 (+ 399us) server_negotiation.cc:299] Negotiation successful
0620 20:09:04.628698 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":46}
I20260620 20:09:04.632306 12315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170: Bootstrap starting.
I20260620 20:09:04.634914 12315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:04.635653 12315 log.cc:824] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170: Log is configured to *not* fsync() on all Append() calls
I20260620 20:09:04.638048 12315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170: No bootstrap required, opened a new log
I20260620 20:09:04.641243 12315 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } }
I20260620 20:09:04.641465 12315 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:04.641508 12315 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b6482bf99b04fb6a6efbfa5ccf5b170, State: Initialized, Role: FOLLOWER
I20260620 20:09:04.641973 12315 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } }
I20260620 20:09:04.642496 12321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } } }
I20260620 20:09:04.642637 12321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:09:04.643204 12315 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [sys.catalog]: configured and running, proceeding with master startup.
W20260620 20:09:04.648262 12332 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 20:09:04.648365 12332 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 20:09:04.652882 12163 master_runner.cc:428] Detected that this master 9b6482bf99b04fb6a6efbfa5ccf5b170 is joining an existing cluster
I20260620 20:09:04.652995 12163 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.91.188:41005
I20260620 20:09:04.658692 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 11658
I20260620 20:09:04.667493 11921 heartbeater.cc:503] Master 127.5.91.190:41567 was elected leader, sending a full tablet report...
I20260620 20:09:04.668859 12338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.655927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:34727 (local address 127.0.0.1:47246)
0620 20:09:04.656308 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:04.656335 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:04.656471 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:04.656742 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:04.656745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:04.657361 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:04.657530 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.657538 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.658588 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.658591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.660330 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.660350 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.661330 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.661333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.661435 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.662105 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:04.662180 (+ 75us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:04.663028 (+ 848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:04.665954 (+ 2926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.665958 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.665961 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.666374 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.666787 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.666848 (+ 61us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.666850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.666942 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.667719 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:04.667724 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:04.667849 (+ 125us) client_negotiation.cc:769] Sending connection context
0620 20:09:04.668462 (+ 613us) client_negotiation.cc:240] Negotiation successful
0620 20:09:04.668665 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":138,"threads_started":1}
I20260620 20:09:04.669214 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.656949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39674 (local address 127.5.91.190:41567)
0620 20:09:04.657148 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:04.657154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:04.657989 (+ 835us) server_negotiation.cc:407] Connection header received
0620 20:09:04.658179 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:04.658182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:04.658232 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:04.658283 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:04.659371 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.660389 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.662686 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.663050 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.663138 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.664655 (+ 1517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:04.664660 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:04.664663 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:04.664702 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:04.666962 (+ 2260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.667732 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.667737 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.667739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.667913 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.668237 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.668241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.668242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.668485 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:04.668562 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:04.668928 (+ 366us) server_negotiation.cc:299] Negotiation successful
0620 20:09:04.669065 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260620 20:09:04.669669 12339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.657217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.0.0.1:39674)
0620 20:09:04.657895 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:04.657916 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:04.658052 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:04.658451 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:04.658454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:04.658679 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:04.659223 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.659232 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.660841 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.660844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.662424 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.662503 (+ 79us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.663314 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.663317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.663433 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.663966 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:04.663980 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:04.664438 (+ 458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:04.667145 (+ 2707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.667148 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.667150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.667444 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.668045 (+ 601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.668048 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.668050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.668145 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.668631 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:04.668634 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:04.668739 (+ 105us) client_negotiation.cc:769] Sending connection context
0620 20:09:04.669353 (+ 614us) client_negotiation.cc:240] Negotiation successful
0620 20:09:04.669502 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":155,"threads_started":1}
I20260620 20:09:04.669708 12319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.656385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47246 (local address 127.5.91.189:34727)
0620 20:09:04.656542 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:04.656547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:04.656560 (+ 13us) server_negotiation.cc:407] Connection header received
0620 20:09:04.656602 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:04.656604 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:04.656643 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:04.656708 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:04.657690 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.658464 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.660631 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.661019 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.661144 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.663334 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:04.663339 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:04.663342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:04.663380 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:04.665778 (+ 2398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.666535 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.666540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.666543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.666678 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.667139 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.667144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.667146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.667459 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:04.667584 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:04.669394 (+ 1810us) server_negotiation.cc:299] Negotiation successful
0620 20:09:04.669559 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260620 20:09:04.673434 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:33021
--local_ip_for_outbound_sockets=127.5.91.129
--tserver_master_addrs=127.5.91.190:41567
--webserver_port=37347
--webserver_interface=127.5.91.129
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.129.keytab
--principal=oryx/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:46671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.91.190:41567,127.5.91.189:34727
--tserver_master_addrs=127.5.91.190:41567,127.5.91.189:34727,127.5.91.188:41005 with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 20:09:04.684039 12317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.672907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41590 (local address 127.5.91.188:41005)
0620 20:09:04.673011 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:04.673016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:04.673212 (+ 196us) server_negotiation.cc:407] Connection header received
0620 20:09:04.673358 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:04.673361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:04.673395 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:04.673435 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:04.675737 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.676480 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.677470 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.677738 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.677823 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.679697 (+ 1874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:04.679701 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:04.679704 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:04.679726 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:04.681844 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.682488 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.682492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.682494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.682573 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.682824 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.682827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.682829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.683061 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:04.683142 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:04.683649 (+ 507us) server_negotiation.cc:299] Negotiation successful
0620 20:09:04.683754 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 20:09:04.684538 12339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.672880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:41005 (local address 127.0.0.1:41590)
0620 20:09:04.673159 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:04.673173 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:04.673274 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:04.673571 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:04.673573 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:04.673783 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:04.673898 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.673905 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.676639 (+ 2734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.676642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.677361 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.677369 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.677929 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.677932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.678023 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.678681 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:04.678804 (+ 123us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:04.679502 (+ 698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:04.681961 (+ 2459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.681964 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.681967 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.682387 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.682665 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.682668 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.682670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.682752 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.683186 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:04.683188 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:04.683480 (+ 292us) client_negotiation.cc:769] Sending connection context
0620 20:09:04.684267 (+ 787us) client_negotiation.cc:240] Negotiation successful
0620 20:09:04.684402 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219}
Jun 20 20:09:04 dist-test-slave-l8h4 krb5kdc[11551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986142, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.190@KRBTEST.COM for oryx/127.5.91.188@KRBTEST.COM
I20260620 20:09:04.696532 12154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.678293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:41005 (local address 127.0.0.1:41594)
0620 20:09:04.678473 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:04.678487 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:04.678567 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:04.682781 (+ 4214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:04.682786 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:04.682986 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:04.683114 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.683122 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.684606 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.684610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.685220 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.685231 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.686663 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.686666 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.686770 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.687357 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:04.687387 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:04.689510 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:04.693843 (+ 4333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.693848 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.693850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.694644 (+ 794us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.694984 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.694987 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.694989 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.695084 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.695473 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:04.695476 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:04.695571 (+ 95us) client_negotiation.cc:769] Sending connection context
0620 20:09:04.695655 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 20:09:04.695821 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260620 20:09:04.696545 12341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.678343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41594 (local address 127.5.91.188:41005)
0620 20:09:04.682476 (+ 4133us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:04.682485 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:04.682495 (+ 10us) server_negotiation.cc:407] Connection header received
0620 20:09:04.682569 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:04.682571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:04.682630 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:04.683385 (+ 755us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:04.683542 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.684458 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.685611 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.686512 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.686643 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.691106 (+ 4463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:04.691110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:04.691112 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:04.691141 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:04.693500 (+ 2359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.694756 (+ 1256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.694760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.694762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.694874 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.695165 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.695169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.695170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.695370 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:04.695449 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:04.695716 (+ 267us) server_negotiation.cc:299] Negotiation successful
0620 20:09:04.695843 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":4037,"thread_start_us":188,"threads_started":1}
I20260620 20:09:04.700989 11594 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.91.188:41005: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170"
member_type: NON_VOTER
last_known_addr {
host: "127.5.91.188"
port: 41005
}
attrs {
promote: true
}
}
dest_uuid: "b603c56a55054a1cb6603903b4748a16"
cas_config_opid_index: 5
I20260620 20:09:04.701617 11594 consensus_queue.cc:237] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: NON_VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: true } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: NON_VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: true } }
I20260620 20:09:04.703222 12158 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9b6482bf99b04fb6a6efbfa5ccf5b170 (127.5.91.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: NON_VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: true } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: NON_VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: true } } }
I20260620 20:09:04.705870 12158 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.91.188:41005
I20260620 20:09:04.707846 12288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:04.709731 12288 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 1 FOLLOWER]: Refusing update from remote peer b603c56a55054a1cb6603903b4748a16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260620 20:09:04.711354 12319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.703659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47254 (local address 127.5.91.189:34727)
0620 20:09:04.704217 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:04.704222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:04.704726 (+ 504us) server_negotiation.cc:407] Connection header received
0620 20:09:04.704798 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:04.704801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:04.704866 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:04.705324 (+ 458us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:04.705813 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.706901 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.708894 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.709778 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.710110 (+ 332us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.710321 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:04.711130 (+ 809us) server_negotiation.cc:299] Negotiation successful
0620 20:09:04.711235 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466}
I20260620 20:09:04.711335 12157 sys_catalog.cc:455] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b603c56a55054a1cb6603903b4748a16" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: NON_VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: true } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: NON_VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: true } } }
I20260620 20:09:04.711609 12157 sys_catalog.cc:458] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:04.711759 12157 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: NON_VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:09:04.710858 12154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.703413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:34727 (local address 127.0.0.1:47254)
0620 20:09:04.704349 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:04.704364 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:04.704468 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:04.705563 (+ 1095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:04.705566 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:04.705588 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:04.705687 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.705694 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.707177 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.707181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.708733 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.708748 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.710393 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.710397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.710500 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.710559 (+ 59us) client_negotiation.cc:769] Sending connection context
0620 20:09:04.710628 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 20:09:04.710718 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827}
I20260620 20:09:04.712960 12207 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [term 1 LEARNER]: Refusing update from remote peer b603c56a55054a1cb6603903b4748a16: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260620 20:09:04.714309 12158 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: NON_VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260620 20:09:04.715056 12163 master.cc:537] Master@127.5.91.188:41005 shutting down...
I20260620 20:09:04.716764 12207 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9b6482bf99b04fb6a6efbfa5ccf5b170 (127.5.91.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: NON_VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: true } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: NON_VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: true } } }
I20260620 20:09:04.720791 12236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [sys.catalog]: SysCatalogTable state changed. Reason: New leader b603c56a55054a1cb6603903b4748a16. Latest consensus state: current_term: 1 leader_uuid: "b603c56a55054a1cb6603903b4748a16" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: NON_VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: true } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: NON_VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: true } } }
I20260620 20:09:04.721666 12236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [sys.catalog]: This master's current role is: LEARNER
I20260620 20:09:04.721822 12236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b603c56a55054a1cb6603903b4748a16" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: NON_VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: true } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: NON_VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: true } } }
I20260620 20:09:04.721949 12236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [sys.catalog]: This master's current role is: LEARNER
I20260620 20:09:04.722800 12288 raft_consensus.cc:858] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: NON_VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: NON_VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: true } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: NON_VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: true } }
I20260620 20:09:04.723026 12158 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16: attempting to promote NON_VOTER 0e00b0caef7c4ba7afd5f5a6ced4fada to VOTER
W20260620 20:09:04.723073 12288 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: NON_VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: NON_VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: true } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: NON_VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: true } }
I20260620 20:09:04.723474 12158 consensus_queue.cc:237] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: NON_VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: true } }
I20260620 20:09:04.725363 12206 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [term 1 LEARNER]: Refusing update from remote peer b603c56a55054a1cb6603903b4748a16: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260620 20:09:04.725587 12288 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0e00b0caef7c4ba7afd5f5a6ced4fada (127.5.91.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: NON_VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: true } } }
I20260620 20:09:04.725862 12157 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260620 20:09:04.728298 12321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b603c56a55054a1cb6603903b4748a16. Latest consensus state: current_term: 1 leader_uuid: "b603c56a55054a1cb6603903b4748a16" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: NON_VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: true } } }
I20260620 20:09:04.728431 12321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260620 20:09:04.728520 12321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b603c56a55054a1cb6603903b4748a16" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: NON_VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: true } } }
I20260620 20:09:04.728583 12321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260620 20:09:04.728645 12321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b603c56a55054a1cb6603903b4748a16" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: NON_VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: true } } }
I20260620 20:09:04.728701 12321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [sys.catalog]: This master's current role is: NON_PARTICIPANT
W20260620 20:09:04.729127 11582 consensus_peers.cc:487] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 -> Peer 9b6482bf99b04fb6a6efbfa5ccf5b170 (127.5.91.188:41005): Couldn't send request to peer 9b6482bf99b04fb6a6efbfa5ccf5b170. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260620 20:09:04.729641 12157 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 0e00b0caef7c4ba7afd5f5a6ced4fada (127.5.91.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: NON_VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: true } } }
I20260620 20:09:04.731427 12157 sys_catalog.cc:455] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b603c56a55054a1cb6603903b4748a16" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: NON_VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: true } } }
I20260620 20:09:04.731551 12157 sys_catalog.cc:458] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:04.731680 12157 sys_catalog.cc:455] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b603c56a55054a1cb6603903b4748a16" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: NON_VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: true } } }
I20260620 20:09:04.731958 12157 sys_catalog.cc:458] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:04.732735 12206 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 0e00b0caef7c4ba7afd5f5a6ced4fada (127.5.91.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: NON_VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: true } } }
I20260620 20:09:04.735286 12344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b603c56a55054a1cb6603903b4748a16" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: NON_VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: true } } }
I20260620 20:09:04.735441 12344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:09:04.736388 12163 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260620 20:09:04.736625 12345 mvcc.cc:205] Tried to move back new op lower bound from 7299015242022805504 to 7299015241860521984. Current Snapshot: MvccSnapshot[applied={T|T < 7299015241909608448}]
I20260620 20:09:04.736650 12163 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260620 20:09:04.736809 12163 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170: stopping tablet replica
I20260620 20:09:04.758915 12163 master.cc:552] Master@127.5.91.188:41005 shutdown complete.
I20260620 20:09:04.758980 12163 master_runner.cc:305] Clearing existing system tablet
I20260620 20:09:04.759196 12163 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:04.759253 12163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:04.760494 12163 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 20:09:04.761288 12353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:04.761514 12163 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:04.761634 12163 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 20:09:04.761668 12163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/wal
uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170"
format_stamp: "Formatted at 2026-06-20 20:09:04 on dist-test-slave-l8h4"
server_key: "4713c1129c355303a9ef4f356252473f"
server_key_iv: "2de673447a34e9cca9f79450253f7c0c"
server_key_version: "encryptionkey@0"
I20260620 20:09:04.763262 12163 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:09:04.765940 12163 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260620 20:09:04.766040 12163 log.cc:1196] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260620 20:09:04.766383 12163 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170: Deleting consensus metadata
I20260620 20:09:04.766536 12163 master_runner.cc:315] Copying system tablet from 127.5.91.190:41567
I20260620 20:09:04.767942 12163 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170: tablet copy: Beginning tablet copy session from remote peer at address 127.5.91.190:41567
I20260620 20:09:04.777297 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.768884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39686 (local address 127.5.91.190:41567)
0620 20:09:04.769103 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:04.769108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:04.769260 (+ 152us) server_negotiation.cc:407] Connection header received
0620 20:09:04.769394 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:04.769397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:04.769442 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:04.769509 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:04.770478 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.771025 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.771880 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.772400 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.772570 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.774314 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:04.774318 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:04.774320 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:04.774349 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:04.775696 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.776111 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.776114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.776116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.776186 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.776490 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.776493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.776494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.776685 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:04.776770 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:04.777017 (+ 247us) server_negotiation.cc:299] Negotiation successful
0620 20:09:04.777128 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260620 20:09:04.777318 12359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.768793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.0.0.1:39686)
0620 20:09:04.769169 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:04.769182 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:04.769303 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:04.769550 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:04.769553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:04.769754 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:04.770352 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.770363 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.771156 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.771159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.771764 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.771772 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.772617 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.772622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.772817 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.773633 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:04.773657 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:04.774094 (+ 437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:04.775814 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.775817 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.775819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.776019 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.776265 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.776269 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.776271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.776389 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.776774 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:04.776777 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:04.776873 (+ 96us) client_negotiation.cc:769] Sending connection context
0620 20:09:04.776958 (+ 85us) client_negotiation.cc:240] Negotiation successful
0620 20:09:04.777121 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":116,"threads_started":1}
I20260620 20:09:04.778039 11616 tablet_copy_service.cc:140] P b603c56a55054a1cb6603903b4748a16: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9b6482bf99b04fb6a6efbfa5ccf5b170 ({username='oryx', principal='oryx/127.5.91.188@KRBTEST.COM'} at 127.0.0.1:39686)
I20260620 20:09:04.778172 11616 tablet_copy_service.cc:161] P b603c56a55054a1cb6603903b4748a16: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9b6482bf99b04fb6a6efbfa5ccf5b170 at {username='oryx', principal='oryx/127.5.91.188@KRBTEST.COM'} at 127.0.0.1:39686: session id = 9b6482bf99b04fb6a6efbfa5ccf5b170-00000000000000000000000000000000
I20260620 20:09:04.779278 11616 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 20:09:04.780560 12163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260620 20:09:04.782651 12163 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170: tablet copy: Starting download of 0 data blocks...
I20260620 20:09:04.782848 12163 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170: tablet copy: Starting download of 1 WAL segments...
I20260620 20:09:04.784227 12163 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 20:09:04.785967 11616 tablet_copy_service.cc:342] P b603c56a55054a1cb6603903b4748a16: Request end of tablet copy session 9b6482bf99b04fb6a6efbfa5ccf5b170-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.91.188@KRBTEST.COM'} at 127.0.0.1:39686
I20260620 20:09:04.786072 11616 tablet_copy_service.cc:433] P b603c56a55054a1cb6603903b4748a16: ending tablet copy session 9b6482bf99b04fb6a6efbfa5ccf5b170-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9b6482bf99b04fb6a6efbfa5ccf5b170
I20260620 20:09:04.793125 12163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:04.795079 12363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:04.795079 12360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:04.795650 12361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:04.795800 12163 server_base.cc:1034] running on GCE node
I20260620 20:09:04.796139 12163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:04.796516 12163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:04.797684 12163 hybrid_clock.cc:648] HybridClock initialized: now 1781986144797671 us; error 33 us; skew 500 ppm
Jun 20 20:09:04 dist-test-slave-l8h4 krb5kdc[11551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986144, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:04.799431 12163 init.cc:377] Logged in from keytab as oryx/127.5.91.188@KRBTEST.COM (short username oryx)
I20260620 20:09:04.800076 12163 webserver.cc:460] Webserver started at http://127.5.91.188:34323/ using document root <none> and password file <none>
I20260620 20:09:04.800320 12163 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:04.800360 12163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:04.801774 12163 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 20:09:04.803159 12370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:04.803448 12163 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 20:09:04.803526 12163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/wal
uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170"
format_stamp: "Formatted at 2026-06-20 20:09:04 on dist-test-slave-l8h4"
server_key: "4713c1129c355303a9ef4f356252473f"
server_key_iv: "2de673447a34e9cca9f79450253f7c0c"
server_key_version: "encryptionkey@0"
I20260620 20:09:04.803625 12163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:04.829613 12163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:04.831461 12163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:04.837291 12422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.188:41005 every 8 connection(s)
I20260620 20:09:04.837288 12163 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.188:41005
I20260620 20:09:04.837945 12163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/master-2/data/info.pb
I20260620 20:09:04.839443 12423 sys_catalog.cc:263] Verifying existing consensus state
I20260620 20:09:04.840728 12423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170: Bootstrap starting.
I20260620 20:09:04.847625 12423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 20:09:04.847903 12423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170: Bootstrap complete.
I20260620 20:09:04.848436 12423 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: NON_VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: true } }
I20260620 20:09:04.848810 12423 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9b6482bf99b04fb6a6efbfa5ccf5b170, State: Initialized, Role: LEARNER
I20260620 20:09:04.849037 12423 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: NON_VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: true } }
I20260620 20:09:04.849526 12426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: NON_VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: true } } }
I20260620 20:09:04.849874 12423 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:09:04.849874 12426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [sys.catalog]: This master's current role is: LEARNER
I20260620 20:09:04.854038 12437 catalog_manager.cc:1259] Loaded cluster ID: 81149eaee63b41e9af14e730a116d9b8
I20260620 20:09:04.854107 12437 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170: loading cluster ID for follower catalog manager: success
I20260620 20:09:04.855731 12437 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170: acquiring CA information for follower catalog manager: success
I20260620 20:09:04.856896 12437 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260620 20:09:04.875078 12340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:04.875468 12340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:04.875531 12340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:04.881317 12340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:04.881398 12340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:04.881464 12340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:09:04.887233 12340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:46671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.129.keytab
--principal=oryx/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:33021
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=37347
--tserver_master_addrs=127.5.91.190:41567,127.5.91.189:34727,127.5.91.188:41005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:04.888442 12340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:04.889595 12340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:04.900233 12445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:04.900199 12442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:04.900192 12443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:04.900686 12340 server_base.cc:1034] running on GCE node
I20260620 20:09:04.901113 12340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:04.901773 12340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:04.902961 12340 hybrid_clock.cc:648] HybridClock initialized: now 1781986144902941 us; error 39 us; skew 500 ppm
Jun 20 20:09:04 dist-test-slave-l8h4 krb5kdc[11551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986144, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:04.907320 12340 init.cc:377] Logged in from keytab as oryx/127.5.91.129@KRBTEST.COM (short username oryx)
I20260620 20:09:04.908730 12340 webserver.cc:460] Webserver started at http://127.5.91.129:37347/ using document root <none> and password file <none>
I20260620 20:09:04.909683 12340 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:04.909828 12340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:04.915132 12340 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260620 20:09:04.919064 12452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:04.920682 12340 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260620 20:09:04.920816 12340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "75c3d02cf5a54f0f8dbf93edc270a213"
format_stamp: "Formatted at 2026-06-20 20:09:03 on dist-test-slave-l8h4"
server_key: "96db82f19ae5f2bdd9f46f558d7455c9"
server_key_iv: "7fcc4ca3bb447e78711bf304ca2042b3"
server_key_version: "encryptionkey@0"
I20260620 20:09:04.921393 12340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:04.946923 12340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:04.951660 12340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:04.951835 12340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:04.952466 12340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:04.953377 12340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:04.953429 12340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:04.953471 12340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:04.953485 12340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:04.964339 12340 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:33021
I20260620 20:09:04.964617 12565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:33021 every 8 connection(s)
I20260620 20:09:04.965456 12340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:09:04.965473 12055 heartbeater.cc:503] Master 127.5.91.190:41567 was elected leader, sending a full tablet report...
I20260620 20:09:04.966492 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 12340
I20260620 20:09:04.966699 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 11792
I20260620 20:09:04.973948 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:45725
--local_ip_for_outbound_sockets=127.5.91.130
--tserver_master_addrs=127.5.91.190:41567
--webserver_port=35095
--webserver_interface=127.5.91.130
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.130.keytab
--principal=oryx/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:46671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.91.190:41567,127.5.91.189:34727
--tserver_master_addrs=127.5.91.190:41567,127.5.91.189:34727,127.5.91.188:41005 with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 20:09:04 dist-test-slave-l8h4 krb5kdc[11551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986144, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM
Jun 20 20:09:04 dist-test-slave-l8h4 krb5kdc[11551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986144, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.189@KRBTEST.COM
Jun 20 20:09:04 dist-test-slave-l8h4 krb5kdc[11551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986144, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.188@KRBTEST.COM
I20260620 20:09:04.984126 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.967821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:48381 (local address 127.5.91.190:41567)
0620 20:09:04.967973 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:04.967979 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:04.970080 (+ 2101us) server_negotiation.cc:407] Connection header received
0620 20:09:04.971394 (+ 1314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:04.971398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:04.971458 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:04.971545 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:04.973135 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.973839 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.975620 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.975852 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.975960 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.980072 (+ 4112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:04.980076 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:04.980079 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:04.980108 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:04.981572 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.982270 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.982273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.982275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.982356 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.982780 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.982783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.982785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.983025 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:04.983099 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:04.983804 (+ 705us) server_negotiation.cc:299] Negotiation successful
0620 20:09:04.983957 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 20:09:04.984169 12573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.970804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:53765 (local address 127.5.91.188:41005)
0620 20:09:04.971077 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:04.971080 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:04.973718 (+ 2638us) server_negotiation.cc:407] Connection header received
0620 20:09:04.973802 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:04.973805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:04.973855 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:04.973923 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:04.974521 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.975558 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.976774 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.977391 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.977540 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.980292 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:04.980297 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:04.980300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:04.980330 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:04.982218 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.982753 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.982756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.982758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.982850 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.983159 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.983162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.983164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.983412 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:04.983486 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:04.983852 (+ 366us) server_negotiation.cc:299] Negotiation successful
0620 20:09:04.983992 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":69,"threads_started":1}
I20260620 20:09:04.985316 12319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.970513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:59403 (local address 127.5.91.189:34727)
0620 20:09:04.970630 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:04.970634 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:04.971624 (+ 990us) server_negotiation.cc:407] Connection header received
0620 20:09:04.971728 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:04.971731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:04.971777 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:04.971847 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:04.973993 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.974700 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.976874 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.977288 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.977390 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.979948 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:04.979953 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:04.979955 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:04.979984 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:04.982105 (+ 2121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.982621 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.982624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.982626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.982702 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:04.982970 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:04.982973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:04.982975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:04.983175 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:04.983252 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:04.985058 (+ 1806us) server_negotiation.cc:299] Negotiation successful
0620 20:09:04.985176 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 20:09:04.985698 12570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.969192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.5.91.129:48381)
0620 20:09:04.969910 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:04.969974 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:04.971106 (+ 1132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:04.971869 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:04.971879 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:04.972369 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:04.972974 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.972989 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.973986 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.973989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.975481 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.975491 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.976157 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.976161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.976368 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.977224 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:04.977246 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:04.979875 (+ 2629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:04.981745 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.981752 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.981773 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.982161 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.982482 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.982485 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.982488 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.982674 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.983162 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:04.983169 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:04.983555 (+ 386us) client_negotiation.cc:769] Sending connection context
0620 20:09:04.984372 (+ 817us) client_negotiation.cc:240] Negotiation successful
0620 20:09:04.984534 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":66,"thread_start_us":251,"threads_started":1}
I20260620 20:09:04.985733 12571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.970642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:41005 (local address 127.5.91.129:53765)
0620 20:09:04.973622 (+ 2980us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:04.973635 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:04.973721 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:04.974042 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:04.974045 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:04.974281 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:04.974392 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.974398 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.975637 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.975640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.976637 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.976648 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.977892 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.977895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.977985 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.978473 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:04.978487 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:04.980122 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:04.982340 (+ 2218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.982344 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.982346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.982632 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.982963 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.982966 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.982968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.983056 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.983564 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:04.983567 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:04.983664 (+ 97us) client_negotiation.cc:769] Sending connection context
0620 20:09:04.984366 (+ 702us) client_negotiation.cc:240] Negotiation successful
0620 20:09:04.984609 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2922,"mutex_wait_us":158,"thread_start_us":75,"threads_started":1}
I20260620 20:09:04.986617 12572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:04.969596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:34727 (local address 127.5.91.129:59403)
0620 20:09:04.971535 (+ 1939us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:04.971548 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:04.971643 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:04.971876 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:04.971879 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:04.972377 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:04.972977 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.972991 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.975671 (+ 2680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.975673 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.976722 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:04.976733 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:04.977545 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:04.977548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:04.977641 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:04.978128 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:04.978180 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:04.979698 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:04.982251 (+ 2553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.982254 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.982256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.982521 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.982794 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:04.982796 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:04.982798 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:04.982868 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:04.983288 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:04.983291 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:04.983519 (+ 228us) client_negotiation.cc:769] Sending connection context
0620 20:09:04.983730 (+ 211us) client_negotiation.cc:240] Negotiation successful
0620 20:09:04.984134 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1874,"mutex_wait_us":59,"spinlock_wait_cycles":6912,"thread_start_us":87,"threads_started":1}
I20260620 20:09:04.987680 12567 heartbeater.cc:347] Connected to a master server at 127.5.91.188:41005
I20260620 20:09:04.988037 12566 heartbeater.cc:347] Connected to a master server at 127.5.91.190:41567
I20260620 20:09:04.988116 12567 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:04.988173 12566 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:04.988818 12568 heartbeater.cc:347] Connected to a master server at 127.5.91.189:34727
I20260620 20:09:04.988919 12568 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:04.989154 12567 heartbeater.cc:511] Master 127.5.91.188:41005 requested a full tablet report, sending...
I20260620 20:09:04.989153 12566 heartbeater.cc:511] Master 127.5.91.190:41567 requested a full tablet report, sending...
I20260620 20:09:04.989153 12568 heartbeater.cc:511] Master 127.5.91.189:34727 requested a full tablet report, sending...
I20260620 20:09:04.991034 11594 ts_manager.cc:194] Re-registered known tserver with Master: 75c3d02cf5a54f0f8dbf93edc270a213 (127.5.91.129:33021)
I20260620 20:09:04.991107 12197 ts_manager.cc:194] Registered new tserver with Master: 75c3d02cf5a54f0f8dbf93edc270a213 (127.5.91.129:33021)
I20260620 20:09:04.991107 12386 ts_manager.cc:194] Registered new tserver with Master: 75c3d02cf5a54f0f8dbf93edc270a213 (127.5.91.129:33021)
I20260620 20:09:04.992144 11594 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:48381
W20260620 20:09:05.114974 12574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:05.115284 12574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:05.115347 12574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:05.119704 12574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:05.119794 12574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:05.119890 12574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:09:05.126549 12574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:46671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.130.keytab
--principal=oryx/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:45725
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=35095
--tserver_master_addrs=127.5.91.190:41567,127.5.91.189:34727,127.5.91.188:41005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:05.127732 12574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:05.128811 12574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:05.139793 12579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:05.139796 12582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:05.139796 12580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:05.140237 12574 server_base.cc:1034] running on GCE node
I20260620 20:09:05.140666 12574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:05.141427 12574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:05.142769 12574 hybrid_clock.cc:648] HybridClock initialized: now 1781986145142724 us; error 41 us; skew 500 ppm
Jun 20 20:09:05 dist-test-slave-l8h4 krb5kdc[11551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986145, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:05.146122 12574 init.cc:377] Logged in from keytab as oryx/127.5.91.130@KRBTEST.COM (short username oryx)
I20260620 20:09:05.147243 12574 webserver.cc:460] Webserver started at http://127.5.91.130:35095/ using document root <none> and password file <none>
I20260620 20:09:05.147819 12574 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:05.147908 12574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:05.152407 12574 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 20:09:05.155113 12589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:05.156836 12574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:09:05.156965 12574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "daade419eb9f498994ac15c2bfa4911b"
format_stamp: "Formatted at 2026-06-20 20:09:03 on dist-test-slave-l8h4"
server_key: "02fc4501a12253028345b0a71b8928af"
server_key_iv: "eb8c7a6214e74a49384fddecdadadf80"
server_key_version: "encryptionkey@0"
I20260620 20:09:05.157552 12574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:05.196447 12574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:05.200069 12574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:05.200515 12574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:05.201736 12574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:05.202837 12574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:05.202895 12574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:05.202939 12574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:05.202960 12574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:05.212919 12574 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:45725
I20260620 20:09:05.213109 12702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:45725 every 8 connection(s)
I20260620 20:09:05.214289 12574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:09:05.224879 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 12574
I20260620 20:09:05.225077 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 11926
Jun 20 20:09:05 dist-test-slave-l8h4 krb5kdc[11551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986145, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM
Jun 20 20:09:05 dist-test-slave-l8h4 krb5kdc[11551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986145, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.189@KRBTEST.COM
Jun 20 20:09:05 dist-test-slave-l8h4 krb5kdc[11551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986145, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.188@KRBTEST.COM
I20260620 20:09:05.232808 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:39363
--local_ip_for_outbound_sockets=127.5.91.131
--tserver_master_addrs=127.5.91.190:41567
--webserver_port=35361
--webserver_interface=127.5.91.131
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.131.keytab
--principal=oryx/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:46671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.91.190:41567,127.5.91.189:34727
--tserver_master_addrs=127.5.91.190:41567,127.5.91.189:34727,127.5.91.188:41005 with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 20:09:05.234973 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.218618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:52675 (local address 127.5.91.190:41567)
0620 20:09:05.218737 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:05.218742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:05.218798 (+ 56us) server_negotiation.cc:407] Connection header received
0620 20:09:05.219752 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:05.219755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:05.219814 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:05.219916 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:05.221586 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.222466 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.224014 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.224472 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.224541 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.229466 (+ 4925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:05.229471 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:05.229474 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:05.229510 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:05.231619 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.232391 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.232395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.232397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.232495 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.233646 (+ 1151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.233650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.233652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.233953 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:05.234025 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:05.234660 (+ 635us) server_negotiation.cc:299] Negotiation successful
0620 20:09:05.234805 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 20:09:05.235064 12319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.216784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:57231 (local address 127.5.91.189:34727)
0620 20:09:05.216929 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:05.216934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:05.218526 (+ 1592us) server_negotiation.cc:407] Connection header received
0620 20:09:05.219934 (+ 1408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:05.219938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:05.219989 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:05.220068 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:05.221619 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.222200 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.224055 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.224396 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.224474 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.229487 (+ 5013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:05.229493 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:05.229497 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:05.229537 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:05.232169 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.232962 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.232965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.232967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.233051 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.233405 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.233417 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.233419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.233694 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:05.233772 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:05.234664 (+ 892us) server_negotiation.cc:299] Negotiation successful
0620 20:09:05.234803 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 20:09:05.235160 12573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.219189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:37477 (local address 127.5.91.188:41005)
0620 20:09:05.219323 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:05.219327 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:05.219342 (+ 15us) server_negotiation.cc:407] Connection header received
0620 20:09:05.220164 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:05.220167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:05.220221 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:05.220297 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:05.221619 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.222480 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.223598 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.223825 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.224301 (+ 476us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.229487 (+ 5186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:05.229493 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:05.229497 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:05.229537 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:05.233215 (+ 3678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.233881 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.233885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.233887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.233972 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.234294 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.234297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.234299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.234507 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:05.234571 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:05.234892 (+ 321us) server_negotiation.cc:299] Negotiation successful
0620 20:09:05.235019 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 20:09:05.237845 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.218819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:41005 (local address 127.5.91.130:37477)
0620 20:09:05.219070 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:05.219083 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:05.219575 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:05.220357 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:05.220360 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:05.220846 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:05.221448 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.221455 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.222617 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.222620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.223469 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.223478 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.224171 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.224175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.224371 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.225303 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:05.225325 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:05.229220 (+ 3895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:05.233507 (+ 4287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:05.233510 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:05.233512 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:05.233762 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:05.234077 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:05.234080 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:05.234082 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:05.234190 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:05.234637 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:05.234640 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:05.234717 (+ 77us) client_negotiation.cc:769] Sending connection context
0620 20:09:05.235363 (+ 646us) client_negotiation.cc:240] Negotiation successful
0620 20:09:05.236512 (+ 1149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":810,"spinlock_wait_cycles":19072,"thread_start_us":93,"threads_started":1}
I20260620 20:09:05.238273 12707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.217171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:34727 (local address 127.5.91.130:57231)
0620 20:09:05.218338 (+ 1167us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:05.218402 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:05.219571 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:05.220324 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:05.220335 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:05.220846 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:05.221438 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.221448 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.222354 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.222356 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.223858 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.223869 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.224853 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.224857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.225063 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.225832 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:05.225846 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:05.229110 (+ 3264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:05.232498 (+ 3388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:05.232501 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:05.232503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:05.232838 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:05.233150 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:05.233153 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:05.233155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:05.233249 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:05.233880 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:05.233887 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:05.234335 (+ 448us) client_negotiation.cc:769] Sending connection context
0620 20:09:05.235367 (+ 1032us) client_negotiation.cc:240] Negotiation successful
0620 20:09:05.236363 (+ 996us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":930,"mutex_wait_us":141,"spinlock_wait_cycles":27136,"thread_start_us":558,"threads_started":1}
I20260620 20:09:05.238601 12708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.218458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.5.91.130:52675)
0620 20:09:05.218726 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:05.218739 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:05.219543 (+ 804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:05.220395 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:05.220398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:05.220843 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:05.221426 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.221440 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.222580 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.222583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.223858 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.223869 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.224690 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.224694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.224786 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.225620 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:05.225633 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:05.229151 (+ 3518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:05.231813 (+ 2662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:05.231823 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:05.231846 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:05.232253 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:05.232616 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:05.232619 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:05.232621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:05.232823 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:05.234086 (+ 1263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:05.234088 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:05.234366 (+ 278us) client_negotiation.cc:769] Sending connection context
0620 20:09:05.236025 (+ 1659us) client_negotiation.cc:240] Negotiation successful
0620 20:09:05.236631 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":532,"spinlock_wait_cycles":20480,"thread_start_us":110,"threads_started":1}
I20260620 20:09:05.239741 12704 heartbeater.cc:347] Connected to a master server at 127.5.91.188:41005
I20260620 20:09:05.240015 12704 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:05.240644 12704 heartbeater.cc:511] Master 127.5.91.188:41005 requested a full tablet report, sending...
I20260620 20:09:05.240646 12703 heartbeater.cc:347] Connected to a master server at 127.5.91.190:41567
I20260620 20:09:05.240772 12703 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:05.240926 12703 heartbeater.cc:511] Master 127.5.91.190:41567 requested a full tablet report, sending...
I20260620 20:09:05.241254 12705 heartbeater.cc:347] Connected to a master server at 127.5.91.189:34727
I20260620 20:09:05.241328 12705 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:05.241490 12705 heartbeater.cc:511] Master 127.5.91.189:34727 requested a full tablet report, sending...
I20260620 20:09:05.242018 11594 ts_manager.cc:194] Re-registered known tserver with Master: daade419eb9f498994ac15c2bfa4911b (127.5.91.130:45725)
I20260620 20:09:05.242190 12386 ts_manager.cc:194] Registered new tserver with Master: daade419eb9f498994ac15c2bfa4911b (127.5.91.130:45725)
I20260620 20:09:05.242229 12197 ts_manager.cc:194] Registered new tserver with Master: daade419eb9f498994ac15c2bfa4911b (127.5.91.130:45725)
I20260620 20:09:05.243021 11594 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:52675
I20260620 20:09:05.320621 12573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.317165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41610 (local address 127.5.91.188:41005)
0620 20:09:05.317295 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:05.317300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:05.317441 (+ 141us) server_negotiation.cc:407] Connection header received
0620 20:09:05.317576 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:05.317579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:05.317632 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:05.317714 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:05.317964 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.318556 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.319617 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.320049 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.320148 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.320291 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:05.320400 (+ 109us) server_negotiation.cc:299] Negotiation successful
0620 20:09:05.320482 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 20:09:05.320624 12712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.317071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:41005 (local address 127.0.0.1:41610)
0620 20:09:05.317336 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:05.317350 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:05.317474 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:05.317736 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:05.317739 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:05.317755 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:05.317864 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.317869 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.318659 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.318662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.319493 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.319504 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.320171 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.320174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.320265 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.320289 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 20:09:05.320378 (+ 89us) client_negotiation.cc:240] Negotiation successful
0620 20:09:05.320478 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":89,"threads_started":1}
I20260620 20:09:05.321501 12397 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260620 20:09:05.321892 12426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b603c56a55054a1cb6603903b4748a16. Latest consensus state: current_term: 1 leader_uuid: "b603c56a55054a1cb6603903b4748a16" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: NON_VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: true } } }
I20260620 20:09:05.322063 12426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [sys.catalog]: This master's current role is: LEARNER
W20260620 20:09:05.365569 12710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:05.365875 12710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:05.365919 12710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:05.370558 12710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:05.370646 12710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:05.370712 12710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:09:05.383486 12710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:46671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.131.keytab
--principal=oryx/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:39363
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=35361
--tserver_master_addrs=127.5.91.190:41567,127.5.91.189:34727,127.5.91.188:41005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:05.384680 12710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:05.385782 12710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:05.397526 12720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:05.397606 12717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:05.397578 12710 server_base.cc:1034] running on GCE node
W20260620 20:09:05.397887 12718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:05.398316 12710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:05.399006 12710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:05.400281 12710 hybrid_clock.cc:648] HybridClock initialized: now 1781986145400230 us; error 42 us; skew 500 ppm
Jun 20 20:09:05 dist-test-slave-l8h4 krb5kdc[11551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986145, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:05.403928 12710 init.cc:377] Logged in from keytab as oryx/127.5.91.131@KRBTEST.COM (short username oryx)
I20260620 20:09:05.405879 12710 webserver.cc:460] Webserver started at http://127.5.91.131:35361/ using document root <none> and password file <none>
I20260620 20:09:05.406579 12710 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:05.406673 12710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:05.411070 12710 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 20:09:05.413491 12727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:05.415027 12710 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:09:05.415160 12710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "b728814d629c46d2ac07d56b1dff1e6e"
format_stamp: "Formatted at 2026-06-20 20:09:03 on dist-test-slave-l8h4"
server_key: "05aca4110eb6caf597dc7d733ee327ae"
server_key_iv: "06953f90d74a9b156fa6eefe3fddd1f4"
server_key_version: "encryptionkey@0"
I20260620 20:09:05.415748 12710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:05.438789 12710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:05.441998 12710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:05.442217 12710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:05.442797 12710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:05.444167 12710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:05.444253 12710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:05.444324 12710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:05.444356 12710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:05.457204 12710 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:39363
I20260620 20:09:05.457222 12840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:39363 every 8 connection(s)
I20260620 20:09:05.458453 12710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:09:05.465373 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 12710
Jun 20 20:09:05 dist-test-slave-l8h4 krb5kdc[11551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986145, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.189@KRBTEST.COM
Jun 20 20:09:05 dist-test-slave-l8h4 krb5kdc[11551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986145, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM
Jun 20 20:09:05 dist-test-slave-l8h4 krb5kdc[11551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986145, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.188@KRBTEST.COM
I20260620 20:09:05.479085 12319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.461029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:55823 (local address 127.5.91.189:34727)
0620 20:09:05.461162 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:05.461166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:05.462543 (+ 1377us) server_negotiation.cc:407] Connection header received
0620 20:09:05.463572 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:05.463575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:05.463633 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:05.463707 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:05.465294 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.466032 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.467931 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.468192 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.468293 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.472853 (+ 4560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:05.472859 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:05.472862 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:05.472898 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:05.475913 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.476679 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.476683 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.476684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.476786 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.477209 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.477212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.477214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.477491 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:05.477576 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:05.478769 (+ 1193us) server_negotiation.cc:299] Negotiation successful
0620 20:09:05.478906 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":53}
I20260620 20:09:05.479146 12573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.460576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:33229 (local address 127.5.91.188:41005)
0620 20:09:05.460789 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:05.460794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:05.462374 (+ 1580us) server_negotiation.cc:407] Connection header received
0620 20:09:05.464155 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:05.464158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:05.464215 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:05.464293 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:05.465296 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.466127 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.468120 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.468426 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.468666 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.474206 (+ 5540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:05.474210 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:05.474213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:05.474246 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:05.475798 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.477225 (+ 1427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.477228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.477230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.477303 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.477590 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.477594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.477595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.477801 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:05.477870 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:05.478885 (+ 1015us) server_negotiation.cc:299] Negotiation successful
0620 20:09:05.479016 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260620 20:09:05.480062 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.462663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.5.91.131:43703)
0620 20:09:05.462972 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:05.462985 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:05.463357 (+ 372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:05.464085 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:05.464087 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:05.464548 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:05.465134 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.465146 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.466328 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.466332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.468190 (+ 1858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.468208 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.469064 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.469069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.469192 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.470026 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:05.470039 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:05.472794 (+ 2755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:05.476249 (+ 3455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:05.476253 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:05.476255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:05.476681 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:05.477055 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:05.477058 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:05.477060 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:05.477146 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:05.477619 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:05.477626 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:05.478111 (+ 485us) client_negotiation.cc:769] Sending connection context
0620 20:09:05.478364 (+ 253us) client_negotiation.cc:240] Negotiation successful
0620 20:09:05.478904 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":349,"spinlock_wait_cycles":97536,"thread_start_us":83,"threads_started":1}
I20260620 20:09:05.480320 12845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.461512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:41005 (local address 127.5.91.131:33229)
0620 20:09:05.462185 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:05.462263 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:05.463343 (+ 1080us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:05.464342 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:05.464345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:05.464528 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:05.465128 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.465147 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.466328 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.466332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.467818 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.467828 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.468657 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.468662 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.468787 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.469894 (+ 1107us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:05.469922 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:05.473888 (+ 3966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:05.475979 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:05.475989 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:05.476011 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:05.477123 (+ 1112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:05.477414 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:05.477417 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:05.477419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:05.477496 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:05.477947 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:05.477950 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:05.478110 (+ 160us) client_negotiation.cc:769] Sending connection context
0620 20:09:05.479359 (+ 1249us) client_negotiation.cc:240] Negotiation successful
0620 20:09:05.479522 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":293,"spinlock_wait_cycles":24576,"thread_start_us":164,"threads_started":1}
I20260620 20:09:05.480973 12846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.461512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:34727 (local address 127.5.91.131:55823)
0620 20:09:05.462451 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:05.462465 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:05.463407 (+ 942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:05.463911 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:05.463922 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:05.464534 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:05.465126 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.465147 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.466333 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.466337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.467812 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.467822 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.468407 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.468411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.468782 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.469894 (+ 1112us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:05.470019 (+ 125us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:05.472595 (+ 2576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:05.476028 (+ 3433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:05.476031 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:05.476033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:05.476554 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:05.476887 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:05.476890 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:05.476892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:05.477098 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:05.477620 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:05.477627 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:05.478110 (+ 483us) client_negotiation.cc:769] Sending connection context
0620 20:09:05.478357 (+ 247us) client_negotiation.cc:240] Negotiation successful
0620 20:09:05.480768 (+ 2411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":854,"mutex_wait_us":2363,"spinlock_wait_cycles":99840,"thread_start_us":692,"threads_started":1}
I20260620 20:09:05.481561 12843 heartbeater.cc:347] Connected to a master server at 127.5.91.189:34727
I20260620 20:09:05.481809 12843 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:05.481980 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.462776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:43703 (local address 127.5.91.190:41567)
0620 20:09:05.462917 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:05.462921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:05.463082 (+ 161us) server_negotiation.cc:407] Connection header received
0620 20:09:05.463832 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:05.463835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:05.463932 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:05.464020 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:05.465296 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.466127 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.468395 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.468761 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.468872 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.472970 (+ 4098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:05.472975 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:05.472977 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:05.473014 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:05.476069 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.476828 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.476831 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.476833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.476944 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.477243 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.477246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.477248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.477476 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:05.477565 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:05.481568 (+ 4003us) server_negotiation.cc:299] Negotiation successful
0620 20:09:05.481793 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 20:09:05.482422 12843 heartbeater.cc:511] Master 127.5.91.189:34727 requested a full tablet report, sending...
I20260620 20:09:05.483181 12853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.470997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47258 (local address 127.5.91.189:34727)
0620 20:09:05.471451 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:05.471456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:05.471894 (+ 438us) server_negotiation.cc:407] Connection header received
0620 20:09:05.472083 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:05.472088 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:05.472155 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:05.472296 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:05.473371 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.475021 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.476851 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.477218 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.477320 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.479276 (+ 1956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:05.479280 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:05.479283 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:05.479308 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:05.481013 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.481732 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.481736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.481739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.481840 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.482272 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.482276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.482278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.482488 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:05.482620 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:05.482925 (+ 305us) server_negotiation.cc:299] Negotiation successful
0620 20:09:05.483055 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":353,"thread_start_us":164,"threads_started":1}
I20260620 20:09:05.483451 12841 heartbeater.cc:347] Connected to a master server at 127.5.91.190:41567
I20260620 20:09:05.483647 12841 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:05.483965 12841 heartbeater.cc:511] Master 127.5.91.190:41567 requested a full tablet report, sending...
I20260620 20:09:05.481561 12842 heartbeater.cc:347] Connected to a master server at 127.5.91.188:41005
I20260620 20:09:05.484383 12842 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:05.484669 12842 heartbeater.cc:511] Master 127.5.91.188:41005 requested a full tablet report, sending...
I20260620 20:09:05.485203 11594 ts_manager.cc:194] Re-registered known tserver with Master: b728814d629c46d2ac07d56b1dff1e6e (127.5.91.131:39363)
I20260620 20:09:05.485538 12197 ts_manager.cc:194] Registered new tserver with Master: b728814d629c46d2ac07d56b1dff1e6e (127.5.91.131:39363)
I20260620 20:09:05.486032 12386 ts_manager.cc:194] Registered new tserver with Master: b728814d629c46d2ac07d56b1dff1e6e (127.5.91.131:39363)
I20260620 20:09:05.486174 11594 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:43703
I20260620 20:09:05.496832 12852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.470415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39696 (local address 127.5.91.190:41567)
0620 20:09:05.471398 (+ 983us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:05.471412 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:05.481105 (+ 9693us) server_negotiation.cc:407] Connection header received
0620 20:09:05.481159 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:05.481162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:05.481223 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:05.481303 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:05.486528 (+ 5225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.487406 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.489823 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.490326 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.490457 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.493286 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:05.493293 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:05.493296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:05.493409 (+ 113us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:05.495046 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.495537 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.495541 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.495543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.495631 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.495894 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.495898 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.495899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.496109 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:05.496222 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:05.496499 (+ 277us) server_negotiation.cc:299] Negotiation successful
0620 20:09:05.496645 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":855,"thread_start_us":108,"threads_started":1}
I20260620 20:09:05.496838 12573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.476826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41622 (local address 127.5.91.188:41005)
0620 20:09:05.479276 (+ 2450us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:05.479280 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:05.486421 (+ 7141us) server_negotiation.cc:407] Connection header received
0620 20:09:05.486595 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:05.486598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:05.486646 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:05.486715 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:05.487500 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.488209 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.489781 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.490224 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.490391 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.493359 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:05.493364 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:05.493366 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:05.493406 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:05.495128 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.495588 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.495591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.495592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.495670 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.495918 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.495921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.495923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.496111 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:05.496226 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:05.496555 (+ 329us) server_negotiation.cc:299] Negotiation successful
0620 20:09:05.496672 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2390,"thread_start_us":101,"threads_started":1}
I20260620 20:09:05.500478 11594 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:09:05.503721 11594 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:09:05.506644 12711 sys_catalog.cc:455] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b603c56a55054a1cb6603903b4748a16" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: NON_VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: true } } }
I20260620 20:09:05.506785 12711 sys_catalog.cc:458] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:05.509939 12859 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16: attempting to promote NON_VOTER 9b6482bf99b04fb6a6efbfa5ccf5b170 to VOTER
I20260620 20:09:05.510311 12859 consensus_queue.cc:237] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: false } }
I20260620 20:09:05.511394 12206 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [term 1 FOLLOWER]: Refusing update from remote peer b603c56a55054a1cb6603903b4748a16: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260620 20:09:05.511755 12711 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260620 20:09:05.512189 12396 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 1 LEARNER]: Refusing update from remote peer b603c56a55054a1cb6603903b4748a16: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260620 20:09:05.512511 12711 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260620 20:09:05.513823 12711 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9b6482bf99b04fb6a6efbfa5ccf5b170 (127.5.91.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: false } } }
I20260620 20:09:05.513823 12206 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9b6482bf99b04fb6a6efbfa5ccf5b170 (127.5.91.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: false } } }
I20260620 20:09:05.514847 12396 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9b6482bf99b04fb6a6efbfa5ccf5b170 (127.5.91.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: false } } }
I20260620 20:09:05.515329 12711 sys_catalog.cc:455] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b603c56a55054a1cb6603903b4748a16" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: false } } }
I20260620 20:09:05.515446 12711 sys_catalog.cc:458] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:05.515565 12711 sys_catalog.cc:455] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b603c56a55054a1cb6603903b4748a16" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: false } } }
I20260620 20:09:05.515630 12711 sys_catalog.cc:458] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:05.515734 12711 sys_catalog.cc:455] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b603c56a55054a1cb6603903b4748a16" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: false } } }
I20260620 20:09:05.515801 12711 sys_catalog.cc:458] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16 [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:05.516144 12869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b603c56a55054a1cb6603903b4748a16" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: false } } }
I20260620 20:09:05.516271 12869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:09:05.517408 12426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b603c56a55054a1cb6603903b4748a16" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: false } } }
I20260620 20:09:05.517524 12426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:09:05.525830 12712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.519819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:39363 (local address 127.0.0.1:52546)
0620 20:09:05.520326 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:05.520342 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:05.520488 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:05.521396 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:05.521403 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:05.521421 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:05.521513 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.521518 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.522887 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.522890 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.524118 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.524127 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.525381 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.525385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.525495 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.525522 (+ 27us) client_negotiation.cc:769] Sending connection context
0620 20:09:05.525598 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 20:09:05.525678 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":1}
I20260620 20:09:05.526031 12871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.520013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52546 (local address 127.5.91.131:39363)
0620 20:09:05.520706 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:05.520718 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:05.520743 (+ 25us) server_negotiation.cc:407] Connection header received
0620 20:09:05.520825 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:05.520831 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:05.521066 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:05.521401 (+ 335us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:05.521640 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.522739 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.524256 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.525262 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.525375 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.525535 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:05.525750 (+ 215us) server_negotiation.cc:299] Negotiation successful
0620 20:09:05.525873 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":156,"threads_started":1}
I20260620 20:09:05.527784 12874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.520533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50378 (local address 127.5.91.129:33021)
0620 20:09:05.521114 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:05.521119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:05.522031 (+ 912us) server_negotiation.cc:407] Connection header received
0620 20:09:05.522128 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:05.522158 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:05.522330 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:05.522515 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:05.522996 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.524110 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.526365 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.527255 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.527352 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.527502 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:05.527579 (+ 77us) server_negotiation.cc:299] Negotiation successful
0620 20:09:05.527670 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":301,"threads_started":1}
I20260620 20:09:05.527776 12870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.520348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:33021 (local address 127.0.0.1:50378)
0620 20:09:05.521895 (+ 1547us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:05.521907 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:05.522007 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:05.522722 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:05.522725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:05.522743 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:05.522827 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.522833 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.524256 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.524259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.525649 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.525659 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.527365 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.527369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.527453 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.527476 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 20:09:05.527550 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 20:09:05.527627 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1491,"mutex_wait_us":154,"thread_start_us":104,"threads_started":1}
I20260620 20:09:05.529052 12775 tablet_service.cc:1467] Processing CreateTablet for tablet 10e250778d22417493462a4df034876c (DEFAULT_TABLE table=test-table [id=1e978542a993430291d0477a221055fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:09:05.530100 12775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10e250778d22417493462a4df034876c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:05.530287 12500 tablet_service.cc:1467] Processing CreateTablet for tablet 10e250778d22417493462a4df034876c (DEFAULT_TABLE table=test-table [id=1e978542a993430291d0477a221055fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:09:05.531375 12500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10e250778d22417493462a4df034876c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:05.531548 12873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.520578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36874 (local address 127.5.91.130:45725)
0620 20:09:05.520946 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:05.520958 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:05.521761 (+ 803us) server_negotiation.cc:407] Connection header received
0620 20:09:05.521838 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:05.521842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:05.522085 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:05.522271 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:05.522742 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.524879 (+ 2137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.526396 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.530647 (+ 4251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.530771 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.531034 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:05.531288 (+ 254us) server_negotiation.cc:299] Negotiation successful
0620 20:09:05.531394 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":90,"threads_started":1}
I20260620 20:09:05.531550 12872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.520014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:45725 (local address 127.0.0.1:36874)
0620 20:09:05.521612 (+ 1598us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:05.521627 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:05.521726 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:05.522419 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:05.522422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:05.522439 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:05.522541 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.522596 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.524998 (+ 2402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.525002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.526236 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.526246 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.530899 (+ 4653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.530904 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.531090 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.531120 (+ 30us) client_negotiation.cc:769] Sending connection context
0620 20:09:05.531206 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 20:09:05.531292 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1535,"thread_start_us":143,"threads_started":1}
I20260620 20:09:05.535089 12637 tablet_service.cc:1467] Processing CreateTablet for tablet 10e250778d22417493462a4df034876c (DEFAULT_TABLE table=test-table [id=1e978542a993430291d0477a221055fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:09:05.536231 12637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10e250778d22417493462a4df034876c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:05.538820 12875 tablet_bootstrap.cc:492] T 10e250778d22417493462a4df034876c P b728814d629c46d2ac07d56b1dff1e6e: Bootstrap starting.
I20260620 20:09:05.541067 12875 tablet_bootstrap.cc:654] T 10e250778d22417493462a4df034876c P b728814d629c46d2ac07d56b1dff1e6e: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:05.542544 12876 tablet_bootstrap.cc:492] T 10e250778d22417493462a4df034876c P 75c3d02cf5a54f0f8dbf93edc270a213: Bootstrap starting.
I20260620 20:09:05.542629 12875 log.cc:824] T 10e250778d22417493462a4df034876c P b728814d629c46d2ac07d56b1dff1e6e: Log is configured to *not* fsync() on all Append() calls
I20260620 20:09:05.545198 12876 tablet_bootstrap.cc:654] T 10e250778d22417493462a4df034876c P 75c3d02cf5a54f0f8dbf93edc270a213: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:05.545396 12875 tablet_bootstrap.cc:492] T 10e250778d22417493462a4df034876c P b728814d629c46d2ac07d56b1dff1e6e: No bootstrap required, opened a new log
I20260620 20:09:05.545773 12875 ts_tablet_manager.cc:1397] T 10e250778d22417493462a4df034876c P b728814d629c46d2ac07d56b1dff1e6e: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260620 20:09:05.546546 12878 tablet_bootstrap.cc:492] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b: Bootstrap starting.
I20260620 20:09:05.546728 12876 log.cc:824] T 10e250778d22417493462a4df034876c P 75c3d02cf5a54f0f8dbf93edc270a213: Log is configured to *not* fsync() on all Append() calls
I20260620 20:09:05.548893 12876 tablet_bootstrap.cc:492] T 10e250778d22417493462a4df034876c P 75c3d02cf5a54f0f8dbf93edc270a213: No bootstrap required, opened a new log
I20260620 20:09:05.549101 12876 ts_tablet_manager.cc:1397] T 10e250778d22417493462a4df034876c P 75c3d02cf5a54f0f8dbf93edc270a213: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260620 20:09:05.549934 12878 tablet_bootstrap.cc:654] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:05.550925 12878 log.cc:824] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b: Log is configured to *not* fsync() on all Append() calls
I20260620 20:09:05.551734 12875 raft_consensus.cc:348] T 10e250778d22417493462a4df034876c P b728814d629c46d2ac07d56b1dff1e6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b728814d629c46d2ac07d56b1dff1e6e" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39363 } } peers { permanent_uuid: "daade419eb9f498994ac15c2bfa4911b" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45725 } } peers { permanent_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 33021 } }
I20260620 20:09:05.552007 12875 raft_consensus.cc:374] T 10e250778d22417493462a4df034876c P b728814d629c46d2ac07d56b1dff1e6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:05.552063 12875 raft_consensus.cc:729] T 10e250778d22417493462a4df034876c P b728814d629c46d2ac07d56b1dff1e6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b728814d629c46d2ac07d56b1dff1e6e, State: Initialized, Role: FOLLOWER
I20260620 20:09:05.552561 12875 consensus_queue.cc:260] T 10e250778d22417493462a4df034876c P b728814d629c46d2ac07d56b1dff1e6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b728814d629c46d2ac07d56b1dff1e6e" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39363 } } peers { permanent_uuid: "daade419eb9f498994ac15c2bfa4911b" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45725 } } peers { permanent_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 33021 } }
I20260620 20:09:05.552656 12876 raft_consensus.cc:348] T 10e250778d22417493462a4df034876c P 75c3d02cf5a54f0f8dbf93edc270a213 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b728814d629c46d2ac07d56b1dff1e6e" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39363 } } peers { permanent_uuid: "daade419eb9f498994ac15c2bfa4911b" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45725 } } peers { permanent_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 33021 } }
I20260620 20:09:05.552911 12876 raft_consensus.cc:374] T 10e250778d22417493462a4df034876c P 75c3d02cf5a54f0f8dbf93edc270a213 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:05.552964 12876 raft_consensus.cc:729] T 10e250778d22417493462a4df034876c P 75c3d02cf5a54f0f8dbf93edc270a213 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75c3d02cf5a54f0f8dbf93edc270a213, State: Initialized, Role: FOLLOWER
I20260620 20:09:05.553404 12841 heartbeater.cc:503] Master 127.5.91.190:41567 was elected leader, sending a full tablet report...
I20260620 20:09:05.553364 12876 consensus_queue.cc:260] T 10e250778d22417493462a4df034876c P 75c3d02cf5a54f0f8dbf93edc270a213 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b728814d629c46d2ac07d56b1dff1e6e" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39363 } } peers { permanent_uuid: "daade419eb9f498994ac15c2bfa4911b" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45725 } } peers { permanent_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 33021 } }
I20260620 20:09:05.553730 12875 ts_tablet_manager.cc:1428] T 10e250778d22417493462a4df034876c P b728814d629c46d2ac07d56b1dff1e6e: Time spent starting tablet: real 0.008s user 0.005s sys 0.005s
I20260620 20:09:05.554399 12876 ts_tablet_manager.cc:1428] T 10e250778d22417493462a4df034876c P 75c3d02cf5a54f0f8dbf93edc270a213: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260620 20:09:05.554683 12878 tablet_bootstrap.cc:492] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b: No bootstrap required, opened a new log
I20260620 20:09:05.555028 12878 ts_tablet_manager.cc:1397] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.005s
I20260620 20:09:05.555218 12566 heartbeater.cc:503] Master 127.5.91.190:41567 was elected leader, sending a full tablet report...
I20260620 20:09:05.561416 12878 raft_consensus.cc:348] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b728814d629c46d2ac07d56b1dff1e6e" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39363 } } peers { permanent_uuid: "daade419eb9f498994ac15c2bfa4911b" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45725 } } peers { permanent_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 33021 } }
I20260620 20:09:05.561734 12878 raft_consensus.cc:374] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:05.561791 12878 raft_consensus.cc:729] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daade419eb9f498994ac15c2bfa4911b, State: Initialized, Role: FOLLOWER
I20260620 20:09:05.562634 12878 consensus_queue.cc:260] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b728814d629c46d2ac07d56b1dff1e6e" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39363 } } peers { permanent_uuid: "daade419eb9f498994ac15c2bfa4911b" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45725 } } peers { permanent_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 33021 } }
I20260620 20:09:05.563714 12703 heartbeater.cc:503] Master 127.5.91.190:41567 was elected leader, sending a full tablet report...
I20260620 20:09:05.564495 12878 ts_tablet_manager.cc:1428] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260620 20:09:05.667059 12883 raft_consensus.cc:482] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:09:05.667543 12883 raft_consensus.cc:504] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b728814d629c46d2ac07d56b1dff1e6e" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39363 } } peers { permanent_uuid: "daade419eb9f498994ac15c2bfa4911b" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45725 } } peers { permanent_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 33021 } }
I20260620 20:09:05.669440 12883 leader_election.cc:288] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b728814d629c46d2ac07d56b1dff1e6e (127.5.91.131:39363), 75c3d02cf5a54f0f8dbf93edc270a213 (127.5.91.129:33021)
I20260620 20:09:05.673554 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.669631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:33021 (local address 127.5.91.130:36883)
0620 20:09:05.669751 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:05.669784 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:05.669929 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:05.670322 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:05.670325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:05.670344 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:05.670468 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.670474 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.671185 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.671189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.672008 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.672016 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.672915 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.672919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.673165 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.673195 (+ 30us) client_negotiation.cc:769] Sending connection context
0620 20:09:05.673278 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 20:09:05.673373 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260620 20:09:05.673537 12874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.669946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:36883 (local address 127.5.91.129:33021)
0620 20:09:05.670064 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:05.670068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:05.670080 (+ 12us) server_negotiation.cc:407] Connection header received
0620 20:09:05.670125 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:05.670127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:05.670220 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:05.670298 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:05.670572 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.671081 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.672129 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.672733 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.672905 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.673043 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:05.673309 (+ 266us) server_negotiation.cc:299] Negotiation successful
0620 20:09:05.673410 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 20:09:05.673904 12871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.669914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:60833 (local address 127.5.91.131:39363)
0620 20:09:05.670041 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:05.670047 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:05.670171 (+ 124us) server_negotiation.cc:407] Connection header received
0620 20:09:05.670353 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:05.670357 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:05.670407 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:05.670473 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:05.670773 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.671360 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.672417 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.673168 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.673299 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.673358 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:05.673555 (+ 197us) server_negotiation.cc:299] Negotiation successful
0620 20:09:05.673670 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 20:09:05.673938 12707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.669631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:39363 (local address 127.5.91.130:60833)
0620 20:09:05.670023 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:05.670041 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:05.670235 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:05.670501 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:05.670504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:05.670521 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:05.670664 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.670670 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.671494 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.671497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.672279 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:05.672288 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.673301 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.673305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:05.673421 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.673444 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 20:09:05.673528 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 20:09:05.673626 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":125}
I20260620 20:09:05.674717 12520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10e250778d22417493462a4df034876c" candidate_uuid: "daade419eb9f498994ac15c2bfa4911b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" is_pre_election: true
I20260620 20:09:05.674744 12795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10e250778d22417493462a4df034876c" candidate_uuid: "daade419eb9f498994ac15c2bfa4911b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b728814d629c46d2ac07d56b1dff1e6e" is_pre_election: true
I20260620 20:09:05.675117 12520 raft_consensus.cc:2454] T 10e250778d22417493462a4df034876c P 75c3d02cf5a54f0f8dbf93edc270a213 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate daade419eb9f498994ac15c2bfa4911b in term 0.
I20260620 20:09:05.675117 12795 raft_consensus.cc:2454] T 10e250778d22417493462a4df034876c P b728814d629c46d2ac07d56b1dff1e6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate daade419eb9f498994ac15c2bfa4911b in term 0.
I20260620 20:09:05.675650 12590 leader_election.cc:302] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 75c3d02cf5a54f0f8dbf93edc270a213, b728814d629c46d2ac07d56b1dff1e6e, daade419eb9f498994ac15c2bfa4911b; no voters:
I20260620 20:09:05.675927 12883 raft_consensus.cc:2781] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:09:05.676012 12883 raft_consensus.cc:482] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:09:05.676054 12883 raft_consensus.cc:3037] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:05.677937 12883 raft_consensus.cc:504] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b728814d629c46d2ac07d56b1dff1e6e" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39363 } } peers { permanent_uuid: "daade419eb9f498994ac15c2bfa4911b" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45725 } } peers { permanent_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 33021 } }
I20260620 20:09:05.678369 12883 leader_election.cc:288] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b [CANDIDATE]: Term 1 election: Requested vote from peers b728814d629c46d2ac07d56b1dff1e6e (127.5.91.131:39363), 75c3d02cf5a54f0f8dbf93edc270a213 (127.5.91.129:33021)
I20260620 20:09:05.678807 12520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10e250778d22417493462a4df034876c" candidate_uuid: "daade419eb9f498994ac15c2bfa4911b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75c3d02cf5a54f0f8dbf93edc270a213"
I20260620 20:09:05.678928 12520 raft_consensus.cc:3037] T 10e250778d22417493462a4df034876c P 75c3d02cf5a54f0f8dbf93edc270a213 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:05.678996 12795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10e250778d22417493462a4df034876c" candidate_uuid: "daade419eb9f498994ac15c2bfa4911b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b728814d629c46d2ac07d56b1dff1e6e"
I20260620 20:09:05.679134 12795 raft_consensus.cc:3037] T 10e250778d22417493462a4df034876c P b728814d629c46d2ac07d56b1dff1e6e [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:05.680097 12520 raft_consensus.cc:2454] T 10e250778d22417493462a4df034876c P 75c3d02cf5a54f0f8dbf93edc270a213 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate daade419eb9f498994ac15c2bfa4911b in term 1.
I20260620 20:09:05.680248 12795 raft_consensus.cc:2454] T 10e250778d22417493462a4df034876c P b728814d629c46d2ac07d56b1dff1e6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate daade419eb9f498994ac15c2bfa4911b in term 1.
I20260620 20:09:05.680428 12590 leader_election.cc:302] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75c3d02cf5a54f0f8dbf93edc270a213, daade419eb9f498994ac15c2bfa4911b; no voters:
I20260620 20:09:05.680649 12883 raft_consensus.cc:2781] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:09:05.680824 12883 raft_consensus.cc:686] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b [term 1 LEADER]: Becoming Leader. State: Replica: daade419eb9f498994ac15c2bfa4911b, State: Running, Role: LEADER
I20260620 20:09:05.681226 12883 consensus_queue.cc:237] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b728814d629c46d2ac07d56b1dff1e6e" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39363 } } peers { permanent_uuid: "daade419eb9f498994ac15c2bfa4911b" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45725 } } peers { permanent_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 33021 } }
I20260620 20:09:05.685822 11596 catalog_manager.cc:5495] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b reported cstate change: term changed from 0 to 1, leader changed from <none> to daade419eb9f498994ac15c2bfa4911b (127.5.91.130). New cstate: current_term: 1 leader_uuid: "daade419eb9f498994ac15c2bfa4911b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b728814d629c46d2ac07d56b1dff1e6e" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "daade419eb9f498994ac15c2bfa4911b" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 33021 } health_report { overall_health: UNKNOWN } } }
W20260620 20:09:05.710036 12844 tablet.cc:2369] T 10e250778d22417493462a4df034876c P b728814d629c46d2ac07d56b1dff1e6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 20:09:05.717473 12706 tablet.cc:2369] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 20:09:05.717624 12569 tablet.cc:2369] T 10e250778d22417493462a4df034876c P 75c3d02cf5a54f0f8dbf93edc270a213: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:09:05.730798 12873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.727345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36884 (local address 127.5.91.130:45725)
0620 20:09:05.727480 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:05.727485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:05.727499 (+ 14us) server_negotiation.cc:407] Connection header received
0620 20:09:05.727580 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:05.727582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:05.727638 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:05.727729 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:09:05.727976 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.728502 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.729344 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.729561 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.729643 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.729933 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:09:05.730429 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:09:05.730543 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:05.730602 (+ 59us) server_negotiation.cc:299] Negotiation successful
0620 20:09:05.730661 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 20:09:05.737303 12795 raft_consensus.cc:1261] T 10e250778d22417493462a4df034876c P b728814d629c46d2ac07d56b1dff1e6e [term 1 FOLLOWER]: Refusing update from remote peer daade419eb9f498994ac15c2bfa4911b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:09:05.737303 12520 raft_consensus.cc:1261] T 10e250778d22417493462a4df034876c P 75c3d02cf5a54f0f8dbf93edc270a213 [term 1 FOLLOWER]: Refusing update from remote peer daade419eb9f498994ac15c2bfa4911b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:09:05.738189 12884 consensus_queue.cc:1035] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b728814d629c46d2ac07d56b1dff1e6e" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:09:05.738633 12883 consensus_queue.cc:1035] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b [LEADER]: Connected to new peer: Peer: permanent_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:09:05.751770 12888 mvcc.cc:205] Tried to move back new op lower bound from 7299015252931956736 to 7299015252713332736. Current Snapshot: MvccSnapshot[applied={T|T < 7299015252931956736}]
I20260620 20:09:05.762499 11596 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:09:05.762703 11596 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:09:05.769408 11596 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b603c56a55054a1cb6603903b4748a16: Sending DeleteTablet for 3 replicas of tablet 10e250778d22417493462a4df034876c
I20260620 20:09:05.770910 12775 tablet_service.cc:1514] Processing DeleteTablet for tablet 10e250778d22417493462a4df034876c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:09:05 UTC) from {username='oryx', principal='oryx/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:52546
I20260620 20:09:05.771078 12500 tablet_service.cc:1514] Processing DeleteTablet for tablet 10e250778d22417493462a4df034876c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:09:05 UTC) from {username='oryx', principal='oryx/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:50378
I20260620 20:09:05.771689 12637 tablet_service.cc:1514] Processing DeleteTablet for tablet 10e250778d22417493462a4df034876c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:09:05 UTC) from {username='oryx', principal='oryx/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:36874
I20260620 20:09:05.772200 12898 tablet_replica.cc:330] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b: stopping tablet replica
I20260620 20:09:05.772457 12896 tablet_replica.cc:330] T 10e250778d22417493462a4df034876c P b728814d629c46d2ac07d56b1dff1e6e: stopping tablet replica
I20260620 20:09:05.772670 12898 raft_consensus.cc:2229] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b [term 1 LEADER]: Raft consensus shutting down.
I20260620 20:09:05.772766 12896 raft_consensus.cc:2229] T 10e250778d22417493462a4df034876c P b728814d629c46d2ac07d56b1dff1e6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:09:05.773157 12896 raft_consensus.cc:2258] T 10e250778d22417493462a4df034876c P b728814d629c46d2ac07d56b1dff1e6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:09:05.773168 12898 raft_consensus.cc:2258] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:09:05.774349 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 11564
I20260620 20:09:05.776471 12896 ts_tablet_manager.cc:1905] T 10e250778d22417493462a4df034876c P b728814d629c46d2ac07d56b1dff1e6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:09:05.777036 12898 ts_tablet_manager.cc:1905] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:09:05.780839 12897 tablet_replica.cc:330] T 10e250778d22417493462a4df034876c P 75c3d02cf5a54f0f8dbf93edc270a213: stopping tablet replica
I20260620 20:09:05.781225 12897 raft_consensus.cc:2229] T 10e250778d22417493462a4df034876c P 75c3d02cf5a54f0f8dbf93edc270a213 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:09:05.781375 12898 ts_tablet_manager.cc:1918] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:09:05.781528 12898 log.cc:1196] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/wal/wals/10e250778d22417493462a4df034876c
I20260620 20:09:05.781584 12897 raft_consensus.cc:2258] T 10e250778d22417493462a4df034876c P 75c3d02cf5a54f0f8dbf93edc270a213 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:09:05.782178 12898 ts_tablet_manager.cc:1939] T 10e250778d22417493462a4df034876c P daade419eb9f498994ac15c2bfa4911b: Deleting consensus metadata
I20260620 20:09:05.783177 12897 ts_tablet_manager.cc:1905] T 10e250778d22417493462a4df034876c P 75c3d02cf5a54f0f8dbf93edc270a213: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:09:05.786123 12897 ts_tablet_manager.cc:1918] T 10e250778d22417493462a4df034876c P 75c3d02cf5a54f0f8dbf93edc270a213: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:09:05.786257 12897 log.cc:1196] T 10e250778d22417493462a4df034876c P 75c3d02cf5a54f0f8dbf93edc270a213: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-0/wal/wals/10e250778d22417493462a4df034876c
I20260620 20:09:05.786579 12897 ts_tablet_manager.cc:1939] T 10e250778d22417493462a4df034876c P 75c3d02cf5a54f0f8dbf93edc270a213: Deleting consensus metadata
I20260620 20:09:05.787552 12896 ts_tablet_manager.cc:1918] T 10e250778d22417493462a4df034876c P b728814d629c46d2ac07d56b1dff1e6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:09:05.787663 12896 log.cc:1196] T 10e250778d22417493462a4df034876c P b728814d629c46d2ac07d56b1dff1e6e: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/10e250778d22417493462a4df034876c
I20260620 20:09:05.788054 12896 ts_tablet_manager.cc:1939] T 10e250778d22417493462a4df034876c P b728814d629c46d2ac07d56b1dff1e6e: Deleting consensus metadata
W20260620 20:09:05.793097 12456 connection.cc:535] server connection from 127.0.0.1:50378 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 20:09:05.793686 12731 connection.cc:535] server connection from 127.0.0.1:52546 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 20:09:05.793681 12592 connection.cc:535] server connection from 127.0.0.1:36874 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 20:09:05.799136 12853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.772647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47262 (local address 127.5.91.189:34727)
0620 20:09:05.772877 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:05.772882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:05.776391 (+ 3509us) server_negotiation.cc:407] Connection header received
0620 20:09:05.776591 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:05.776595 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:05.776647 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:05.776696 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:05.777793 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.778562 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.779717 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.780045 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.780111 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.791858 (+ 11747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:05.791863 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:05.791866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:05.791903 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:05.793632 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.794802 (+ 1170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.794806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.794808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.794902 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.797614 (+ 2712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.797618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.797620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.797870 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:05.797993 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:05.798832 (+ 839us) server_negotiation.cc:299] Negotiation successful
0620 20:09:05.798966 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260620 20:09:05.807356 12319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.797262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47274 (local address 127.5.91.189:34727)
0620 20:09:05.797397 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:05.797402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:05.797557 (+ 155us) server_negotiation.cc:407] Connection header received
0620 20:09:05.798283 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:05.798286 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:05.798333 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:05.798427 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:05.799084 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.799845 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.800862 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.801202 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.801401 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.803244 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:05.803248 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:05.803251 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:05.803336 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:05.805051 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.805495 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.805499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.805501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.805587 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.805839 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.805841 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.805843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.806381 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:05.806511 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:05.807049 (+ 538us) server_negotiation.cc:299] Negotiation successful
0620 20:09:05.807178 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 20:09:05.811920 12856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.798189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41638 (local address 127.5.91.188:41005)
0620 20:09:05.798312 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:05.798316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:05.798332 (+ 16us) server_negotiation.cc:407] Connection header received
0620 20:09:05.798377 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:05.798380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:05.798450 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:05.798535 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:05.801554 (+ 3019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.802719 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.804507 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.804890 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.805018 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.807989 (+ 2971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:05.807993 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:05.807996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:05.808022 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:05.809915 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.810473 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.810477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.810479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.810557 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.810812 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.810815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.810817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.811053 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:05.811175 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:05.811631 (+ 456us) server_negotiation.cc:299] Negotiation successful
0620 20:09:05.811743 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 20:09:05.813550 12573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:05.772750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41624 (local address 127.5.91.188:41005)
0620 20:09:05.772964 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:05.772968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:05.800538 (+ 27570us) server_negotiation.cc:407] Connection header received
0620 20:09:05.800611 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:05.800614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:05.800662 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:05.800740 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:05.801609 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.802992 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.805626 (+ 2634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:05.805937 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:05.806010 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:05.809017 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:05.809021 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:05.809023 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:05.809050 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:05.811003 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.811406 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.811410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.811411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.811492 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:05.812245 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:05.812250 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:05.812252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:05.812562 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:05.812708 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:05.813211 (+ 503us) server_negotiation.cc:299] Negotiation successful
0620 20:09:05.813365 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260620 20:09:06.756708 12907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:06.756011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.5.91.129:50597)
0620 20:09:06.756471 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:06.756590 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":87,"threads_started":1}
W20260620 20:09:06.757063 12566 heartbeater.cc:644] Failed to heartbeat to 127.5.91.190:41567 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
I20260620 20:09:06.757406 12908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:06.756137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.5.91.131:34205)
0620 20:09:06.756954 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:06.757068 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":740,"thread_start_us":84,"threads_started":1}
W20260620 20:09:06.757653 12841 heartbeater.cc:644] Failed to heartbeat to 127.5.91.190:41567 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
I20260620 20:09:06.759416 12909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:06.758983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.5.91.130:53297)
0620 20:09:06.759218 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:06.759299 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":67,"threads_started":1}
W20260620 20:09:06.759658 12703 heartbeater.cc:644] Failed to heartbeat to 127.5.91.190:41567 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
I20260620 20:09:07.269784 12910 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b603c56a55054a1cb6603903b4748a16)
I20260620 20:09:07.269825 12911 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b603c56a55054a1cb6603903b4748a16)
I20260620 20:09:07.270010 12910 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: false } }
I20260620 20:09:07.270030 12911 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: false } }
I20260620 20:09:07.271023 12911 leader_election.cc:288] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b603c56a55054a1cb6603903b4748a16 (127.5.91.190:41567), 9b6482bf99b04fb6a6efbfa5ccf5b170 (127.5.91.188:41005)
I20260620 20:09:07.271349 12910 leader_election.cc:288] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b603c56a55054a1cb6603903b4748a16 (127.5.91.190:41567), 0e00b0caef7c4ba7afd5f5a6ced4fada (127.5.91.189:34727)
I20260620 20:09:07.273154 12913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.271469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.0.0.1:40136)
0620 20:09:07.272377 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.272986 (+ 609us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":266,"threads_started":1}
W20260620 20:09:07.273696 12373 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
I20260620 20:09:07.274258 12912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.271254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.0.0.1:40134)
0620 20:09:07.273688 (+ 2434us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.273768 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1464,"thread_start_us":56,"threads_started":1}
I20260620 20:09:07.274803 12913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.274560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.0.0.1:40142)
0620 20:09:07.274692 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.274737 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260620 20:09:07.274981 12373 leader_election.cc:334] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b603c56a55054a1cb6603903b4748a16 (127.5.91.190:41567): Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
W20260620 20:09:07.275053 12183 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
I20260620 20:09:07.276242 12912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.275938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.0.0.1:40158)
0620 20:09:07.276118 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.276178 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260620 20:09:07.277370 12914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.271596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:34727 (local address 127.0.0.1:47282)
0620 20:09:07.273249 (+ 1653us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.273273 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:07.273447 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:07.273811 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:07.273814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:07.273832 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:07.273933 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:07.273941 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.275066 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.275069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:07.276029 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:07.276040 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.276893 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.276897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:07.277058 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:07.277083 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 20:09:07.277164 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 20:09:07.277243 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1072,"thread_start_us":63,"threads_started":1}
I20260620 20:09:07.277467 12917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.271871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47282 (local address 127.5.91.189:34727)
0620 20:09:07.273116 (+ 1245us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:07.273121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:07.273483 (+ 362us) server_negotiation.cc:407] Connection header received
0620 20:09:07.273551 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:07.273554 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:07.273605 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:07.273695 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:07.274079 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.274828 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.276169 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.276755 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.276871 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:07.276937 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:07.277200 (+ 263us) server_negotiation.cc:299] Negotiation successful
0620 20:09:07.277291 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1147,"thread_start_us":1008,"threads_started":1}
W20260620 20:09:07.277536 12183 leader_election.cc:334] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b603c56a55054a1cb6603903b4748a16 (127.5.91.190:41567): Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
I20260620 20:09:07.278105 12206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" is_pre_election: true
I20260620 20:09:07.278362 12206 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b6482bf99b04fb6a6efbfa5ccf5b170 in term 1.
I20260620 20:09:07.278635 12916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.271855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41648 (local address 127.5.91.188:41005)
0620 20:09:07.273650 (+ 1795us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:07.273655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:07.273666 (+ 11us) server_negotiation.cc:407] Connection header received
0620 20:09:07.273706 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:07.273708 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:07.273746 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:07.274783 (+ 1037us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:07.274891 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.276003 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.277638 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.278123 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.278254 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:07.278308 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:07.278475 (+ 167us) server_negotiation.cc:299] Negotiation successful
0620 20:09:07.278550 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1730,"thread_start_us":77,"threads_started":1}
I20260620 20:09:07.278658 12915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.271798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:41005 (local address 127.0.0.1:41648)
0620 20:09:07.273370 (+ 1572us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.273384 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:07.273474 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:07.274072 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:07.274075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:07.274093 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:07.274339 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:07.274348 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.276291 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.276295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:07.277353 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:07.277363 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.278258 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.278262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:07.278355 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:07.278377 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 20:09:07.278453 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 20:09:07.278529 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1493,"thread_start_us":86,"threads_started":1}
I20260620 20:09:07.278734 12371 leader_election.cc:302] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e00b0caef7c4ba7afd5f5a6ced4fada, 9b6482bf99b04fb6a6efbfa5ccf5b170; no voters: b603c56a55054a1cb6603903b4748a16
I20260620 20:09:07.278967 12910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260620 20:09:07.279019 12910 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b603c56a55054a1cb6603903b4748a16)
I20260620 20:09:07.279060 12910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 1 FOLLOWER]: Advancing to term 2
I20260620 20:09:07.279277 12396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" is_pre_election: true
I20260620 20:09:07.280251 12910 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: false } }
I20260620 20:09:07.280517 12396 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e00b0caef7c4ba7afd5f5a6ced4fada in current term 2: Already voted for candidate 9b6482bf99b04fb6a6efbfa5ccf5b170 in this term.
I20260620 20:09:07.280736 12910 leader_election.cc:288] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [CANDIDATE]: Term 2 election: Requested vote from peers b603c56a55054a1cb6603903b4748a16 (127.5.91.190:41567), 0e00b0caef7c4ba7afd5f5a6ced4fada (127.5.91.189:34727)
I20260620 20:09:07.281104 12914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.280796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.0.0.1:40168)
0620 20:09:07.280988 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.281043 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260620 20:09:07.281199 12183 leader_election.cc:302] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e00b0caef7c4ba7afd5f5a6ced4fada; no voters: 9b6482bf99b04fb6a6efbfa5ccf5b170, b603c56a55054a1cb6603903b4748a16
I20260620 20:09:07.281389 12911 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [term 1 FOLLOWER]: Advancing to term 2
I20260620 20:09:07.281613 12206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada"
I20260620 20:09:07.281788 12914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.281585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.0.0.1:40176)
0620 20:09:07.281679 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.281729 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 20:09:07.282014 12373 leader_election.cc:334] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b603c56a55054a1cb6603903b4748a16 (127.5.91.190:41567): Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
I20260620 20:09:07.282871 12911 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 20:09:07.283830 12206 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b6482bf99b04fb6a6efbfa5ccf5b170 in term 2.
I20260620 20:09:07.284474 12371 leader_election.cc:302] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e00b0caef7c4ba7afd5f5a6ced4fada, 9b6482bf99b04fb6a6efbfa5ccf5b170; no voters: b603c56a55054a1cb6603903b4748a16
I20260620 20:09:07.284665 12910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 20:09:07.284981 12910 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [term 2 LEADER]: Becoming Leader. State: Replica: 9b6482bf99b04fb6a6efbfa5ccf5b170, State: Running, Role: LEADER
I20260620 20:09:07.285233 12910 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: false } }
I20260620 20:09:07.286262 12920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b6482bf99b04fb6a6efbfa5ccf5b170. Latest consensus state: current_term: 2 leader_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: false } } }
I20260620 20:09:07.286370 12920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:07.286636 12922 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:09:07.289891 12922 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:09:07.290539 12922 catalog_manager.cc:1259] Loaded cluster ID: 81149eaee63b41e9af14e730a116d9b8
I20260620 20:09:07.290596 12922 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:09:07.291278 12922 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:09:07.291698 12922 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170: Loaded TSK: 0
I20260620 20:09:07.292472 12922 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:09:07.301004 12387 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:09:07.305408 12387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:09:07.308763 12914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.308533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.0.0.1:40202)
0620 20:09:07.308637 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.308691 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260620 20:09:07.308902 12206 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [term 2 FOLLOWER]: Refusing update from remote peer 9b6482bf99b04fb6a6efbfa5ccf5b170: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260620 20:09:07.309404 12910 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260620 20:09:07.309412 12914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.309209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.0.0.1:40214)
0620 20:09:07.309306 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.309362 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260620 20:09:07.309619 12373 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 -> Peer b603c56a55054a1cb6603903b4748a16 (127.5.91.190:41567): Couldn't send request to peer b603c56a55054a1cb6603903b4748a16. Status: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260620 20:09:07.311368 12911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b6482bf99b04fb6a6efbfa5ccf5b170. Latest consensus state: current_term: 2 leader_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: false } } }
I20260620 20:09:07.311486 12911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:09:07.313395 12910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: false } } }
I20260620 20:09:07.313535 12910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170 [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:07.313695 12924 mvcc.cc:205] Tried to move back new op lower bound from 7299015259372158976 to 7299015259282493440. Current Snapshot: MvccSnapshot[applied={T|T < 7299015259372158976}]
I20260620 20:09:07.314325 12911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b603c56a55054a1cb6603903b4748a16" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41567 } } peers { permanent_uuid: "0e00b0caef7c4ba7afd5f5a6ced4fada" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 34727 } attrs { promote: false } } peers { permanent_uuid: "9b6482bf99b04fb6a6efbfa5ccf5b170" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41005 } attrs { promote: false } } }
I20260620 20:09:07.314429 12911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e00b0caef7c4ba7afd5f5a6ced4fada [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:09:07.329100 12930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.325123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:39363 (local address 127.0.0.1:52558)
0620 20:09:07.325456 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.325468 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:07.325563 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:07.325816 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:07.325819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:07.325835 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:07.325914 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:07.325918 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.326756 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.326758 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:07.327981 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:07.327992 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.328662 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.328666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:07.328764 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:07.328787 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 20:09:07.328854 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 20:09:07.328942 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":1,"spinlock_wait_cycles":1408,"thread_start_us":101,"threads_started":1}
I20260620 20:09:07.329100 12931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.325237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52558 (local address 127.5.91.131:39363)
0620 20:09:07.325436 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:07.325440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:07.325558 (+ 118us) server_negotiation.cc:407] Connection header received
0620 20:09:07.325645 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:07.325648 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:07.325691 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:07.325757 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:07.326005 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.326666 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.328121 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.328552 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.328647 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:07.328718 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:07.328885 (+ 167us) server_negotiation.cc:299] Negotiation successful
0620 20:09:07.328958 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":52,"threads_started":1}
I20260620 20:09:07.329646 12775 tablet_service.cc:1467] Processing CreateTablet for tablet c110cc0877ab41aeb3540c6d04835a69 (DEFAULT_TABLE table=test-table [id=aaac5b22bebf45f3b50373de884a395d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:09:07.329943 12775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c110cc0877ab41aeb3540c6d04835a69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:07.329772 12914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.324168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:33021 (local address 127.0.0.1:50382)
0620 20:09:07.324320 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.324336 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:07.324433 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:07.326360 (+ 1927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:07.326370 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:07.326387 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:07.326458 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:07.326463 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.327259 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.327263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:07.328035 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:07.328044 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.329428 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.329431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:07.329511 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:07.329534 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 20:09:07.329594 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 20:09:07.329667 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":24}
I20260620 20:09:07.330096 12913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.324412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:45725 (local address 127.0.0.1:36892)
0620 20:09:07.324541 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.324554 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:07.324634 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:07.325479 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:07.325483 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:07.325503 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:07.325631 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:07.325638 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.326909 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.326912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:07.328042 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:07.328048 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.329563 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.329566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:07.329654 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:07.329675 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 20:09:07.329723 (+ 48us) client_negotiation.cc:240] Negotiation successful
0620 20:09:07.329797 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":28,"spinlock_wait_cycles":8320}
I20260620 20:09:07.330410 12928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.324305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50382 (local address 127.5.91.129:33021)
0620 20:09:07.326103 (+ 1798us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:07.326107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:07.326123 (+ 16us) server_negotiation.cc:407] Connection header received
0620 20:09:07.326206 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:07.326209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:07.326254 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:07.326352 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:07.326539 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.327145 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.328722 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.329326 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.329420 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:07.329481 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:07.330194 (+ 713us) server_negotiation.cc:299] Negotiation successful
0620 20:09:07.330286 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":93,"threads_started":1}
I20260620 20:09:07.330571 12929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.324519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36892 (local address 127.5.91.130:45725)
0620 20:09:07.324769 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:07.324774 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:07.324788 (+ 14us) server_negotiation.cc:407] Connection header received
0620 20:09:07.324838 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:07.324841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:07.325152 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:07.325736 (+ 584us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:07.325939 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.326790 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.328153 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.328603 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.329898 (+ 1295us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:07.329955 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:07.330340 (+ 385us) server_negotiation.cc:299] Negotiation successful
0620 20:09:07.330437 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":84,"threads_started":1}
I20260620 20:09:07.331030 12637 tablet_service.cc:1467] Processing CreateTablet for tablet c110cc0877ab41aeb3540c6d04835a69 (DEFAULT_TABLE table=test-table [id=aaac5b22bebf45f3b50373de884a395d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:09:07.331277 12637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c110cc0877ab41aeb3540c6d04835a69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:07.332348 12932 tablet_bootstrap.cc:492] T c110cc0877ab41aeb3540c6d04835a69 P b728814d629c46d2ac07d56b1dff1e6e: Bootstrap starting.
I20260620 20:09:07.332819 12500 tablet_service.cc:1467] Processing CreateTablet for tablet c110cc0877ab41aeb3540c6d04835a69 (DEFAULT_TABLE table=test-table [id=aaac5b22bebf45f3b50373de884a395d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:09:07.333443 12500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c110cc0877ab41aeb3540c6d04835a69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:07.334540 12932 tablet_bootstrap.cc:654] T c110cc0877ab41aeb3540c6d04835a69 P b728814d629c46d2ac07d56b1dff1e6e: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:07.336973 12932 tablet_bootstrap.cc:492] T c110cc0877ab41aeb3540c6d04835a69 P b728814d629c46d2ac07d56b1dff1e6e: No bootstrap required, opened a new log
I20260620 20:09:07.337078 12932 ts_tablet_manager.cc:1397] T c110cc0877ab41aeb3540c6d04835a69 P b728814d629c46d2ac07d56b1dff1e6e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 20:09:07.337603 12932 raft_consensus.cc:348] T c110cc0877ab41aeb3540c6d04835a69 P b728814d629c46d2ac07d56b1dff1e6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 33021 } } peers { permanent_uuid: "daade419eb9f498994ac15c2bfa4911b" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45725 } } peers { permanent_uuid: "b728814d629c46d2ac07d56b1dff1e6e" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39363 } }
I20260620 20:09:07.337728 12932 raft_consensus.cc:374] T c110cc0877ab41aeb3540c6d04835a69 P b728814d629c46d2ac07d56b1dff1e6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:07.337759 12932 raft_consensus.cc:729] T c110cc0877ab41aeb3540c6d04835a69 P b728814d629c46d2ac07d56b1dff1e6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b728814d629c46d2ac07d56b1dff1e6e, State: Initialized, Role: FOLLOWER
I20260620 20:09:07.338006 12932 consensus_queue.cc:260] T c110cc0877ab41aeb3540c6d04835a69 P b728814d629c46d2ac07d56b1dff1e6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 33021 } } peers { permanent_uuid: "daade419eb9f498994ac15c2bfa4911b" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45725 } } peers { permanent_uuid: "b728814d629c46d2ac07d56b1dff1e6e" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39363 } }
I20260620 20:09:07.338311 12935 tablet_bootstrap.cc:492] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213: Bootstrap starting.
I20260620 20:09:07.338387 12932 ts_tablet_manager.cc:1428] T c110cc0877ab41aeb3540c6d04835a69 P b728814d629c46d2ac07d56b1dff1e6e: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260620 20:09:07.339296 12933 tablet_bootstrap.cc:492] T c110cc0877ab41aeb3540c6d04835a69 P daade419eb9f498994ac15c2bfa4911b: Bootstrap starting.
I20260620 20:09:07.339674 12935 tablet_bootstrap.cc:654] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:07.340132 12933 tablet_bootstrap.cc:654] T c110cc0877ab41aeb3540c6d04835a69 P daade419eb9f498994ac15c2bfa4911b: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:07.341588 12933 tablet_bootstrap.cc:492] T c110cc0877ab41aeb3540c6d04835a69 P daade419eb9f498994ac15c2bfa4911b: No bootstrap required, opened a new log
I20260620 20:09:07.341933 12933 ts_tablet_manager.cc:1397] T c110cc0877ab41aeb3540c6d04835a69 P daade419eb9f498994ac15c2bfa4911b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 20:09:07.342221 12842 heartbeater.cc:503] Master 127.5.91.188:41005 was elected leader, sending a full tablet report...
I20260620 20:09:07.342247 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.338843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.5.91.131:41529)
0620 20:09:07.342036 (+ 3193us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.342105 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3116,"thread_start_us":101,"threads_started":1}
I20260620 20:09:07.342590 12933 raft_consensus.cc:348] T c110cc0877ab41aeb3540c6d04835a69 P daade419eb9f498994ac15c2bfa4911b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 33021 } } peers { permanent_uuid: "daade419eb9f498994ac15c2bfa4911b" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45725 } } peers { permanent_uuid: "b728814d629c46d2ac07d56b1dff1e6e" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39363 } }
I20260620 20:09:07.342718 12933 raft_consensus.cc:374] T c110cc0877ab41aeb3540c6d04835a69 P daade419eb9f498994ac15c2bfa4911b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:07.342754 12933 raft_consensus.cc:729] T c110cc0877ab41aeb3540c6d04835a69 P daade419eb9f498994ac15c2bfa4911b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daade419eb9f498994ac15c2bfa4911b, State: Initialized, Role: FOLLOWER
I20260620 20:09:07.342929 12933 consensus_queue.cc:260] T c110cc0877ab41aeb3540c6d04835a69 P daade419eb9f498994ac15c2bfa4911b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 33021 } } peers { permanent_uuid: "daade419eb9f498994ac15c2bfa4911b" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45725 } } peers { permanent_uuid: "b728814d629c46d2ac07d56b1dff1e6e" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39363 } }
I20260620 20:09:07.343380 12933 ts_tablet_manager.cc:1428] T c110cc0877ab41aeb3540c6d04835a69 P daade419eb9f498994ac15c2bfa4911b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260620 20:09:07.345479 12941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.344860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.5.91.130:51961)
0620 20:09:07.345251 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.345337 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":169,"threads_started":1}
I20260620 20:09:07.346962 12704 heartbeater.cc:503] Master 127.5.91.188:41005 was elected leader, sending a full tablet report...
I20260620 20:09:07.347154 12935 tablet_bootstrap.cc:492] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213: No bootstrap required, opened a new log
I20260620 20:09:07.347267 12935 ts_tablet_manager.cc:1397] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20260620 20:09:07.347628 12941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.347134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.5.91.130:42885)
0620 20:09:07.347494 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.347552 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284}
I20260620 20:09:07.346108 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.342911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.5.91.131:58133)
0620 20:09:07.345940 (+ 3029us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.346017 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2880}
I20260620 20:09:07.348721 12935 raft_consensus.cc:348] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 33021 } } peers { permanent_uuid: "daade419eb9f498994ac15c2bfa4911b" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45725 } } peers { permanent_uuid: "b728814d629c46d2ac07d56b1dff1e6e" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39363 } }
I20260620 20:09:07.348848 12935 raft_consensus.cc:374] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:07.348872 12935 raft_consensus.cc:729] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75c3d02cf5a54f0f8dbf93edc270a213, State: Initialized, Role: FOLLOWER
I20260620 20:09:07.349000 12935 consensus_queue.cc:260] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 33021 } } peers { permanent_uuid: "daade419eb9f498994ac15c2bfa4911b" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45725 } } peers { permanent_uuid: "b728814d629c46d2ac07d56b1dff1e6e" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39363 } }
I20260620 20:09:07.349342 12935 ts_tablet_manager.cc:1428] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260620 20:09:07.350595 12943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.350099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.5.91.129:59269)
0620 20:09:07.350430 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.350492 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":200,"threads_started":1}
I20260620 20:09:07.351475 12567 heartbeater.cc:503] Master 127.5.91.188:41005 was elected leader, sending a full tablet report...
I20260620 20:09:07.351785 12943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.351090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.5.91.129:41305)
0620 20:09:07.351265 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.351707 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260620 20:09:07.580752 12942 raft_consensus.cc:482] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:09:07.581051 12942 raft_consensus.cc:504] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 33021 } } peers { permanent_uuid: "daade419eb9f498994ac15c2bfa4911b" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45725 } } peers { permanent_uuid: "b728814d629c46d2ac07d56b1dff1e6e" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39363 } }
I20260620 20:09:07.582166 12942 leader_election.cc:288] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers daade419eb9f498994ac15c2bfa4911b (127.5.91.130:45725), b728814d629c46d2ac07d56b1dff1e6e (127.5.91.131:39363)
I20260620 20:09:07.587849 12929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.582561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:39253 (local address 127.5.91.130:45725)
0620 20:09:07.582695 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:07.582699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:07.582717 (+ 18us) server_negotiation.cc:407] Connection header received
0620 20:09:07.582952 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:07.582956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:07.583008 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:07.583058 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:07.583538 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.584490 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.586279 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.586892 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.587056 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:07.587108 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:07.587514 (+ 406us) server_negotiation.cc:299] Negotiation successful
0620 20:09:07.587716 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 20:09:07.587837 12931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.582803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:35919 (local address 127.5.91.131:39363)
0620 20:09:07.582932 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:07.582936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:07.583072 (+ 136us) server_negotiation.cc:407] Connection header received
0620 20:09:07.583165 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:07.583167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:07.583210 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:07.583273 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:07.583577 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.584490 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.586279 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.586930 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.587056 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:07.587108 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:07.587527 (+ 419us) server_negotiation.cc:299] Negotiation successful
0620 20:09:07.587682 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 20:09:07.587831 12943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.582446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:45725 (local address 127.5.91.129:39253)
0620 20:09:07.582608 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.582626 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:07.582809 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:07.583192 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:07.583194 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:07.583209 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:07.583314 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:07.583321 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.584634 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.584638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:07.586113 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:07.586123 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.587103 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.587107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:07.587261 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:07.587296 (+ 35us) client_negotiation.cc:769] Sending connection context
0620 20:09:07.587406 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 20:09:07.587471 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":184}
I20260620 20:09:07.587850 12944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.582702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:39363 (local address 127.5.91.129:35919)
0620 20:09:07.582997 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.583009 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:07.583091 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:07.583348 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:07.583351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:07.583366 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:07.583449 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:07.583454 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.584634 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.584638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:07.586111 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:07.586123 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.587103 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.587107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:07.587252 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:07.587290 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 20:09:07.587372 (+ 82us) client_negotiation.cc:240] Negotiation successful
0620 20:09:07.587453 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":72,"thread_start_us":118,"threads_started":1}
I20260620 20:09:07.588726 12657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c110cc0877ab41aeb3540c6d04835a69" candidate_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "daade419eb9f498994ac15c2bfa4911b" is_pre_election: true
I20260620 20:09:07.588780 12795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c110cc0877ab41aeb3540c6d04835a69" candidate_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b728814d629c46d2ac07d56b1dff1e6e" is_pre_election: true
I20260620 20:09:07.588989 12795 raft_consensus.cc:2454] T c110cc0877ab41aeb3540c6d04835a69 P b728814d629c46d2ac07d56b1dff1e6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75c3d02cf5a54f0f8dbf93edc270a213 in term 0.
I20260620 20:09:07.588995 12657 raft_consensus.cc:2454] T c110cc0877ab41aeb3540c6d04835a69 P daade419eb9f498994ac15c2bfa4911b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75c3d02cf5a54f0f8dbf93edc270a213 in term 0.
I20260620 20:09:07.589543 12456 leader_election.cc:302] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75c3d02cf5a54f0f8dbf93edc270a213, b728814d629c46d2ac07d56b1dff1e6e; no voters:
I20260620 20:09:07.589879 12942 raft_consensus.cc:2781] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:09:07.589957 12942 raft_consensus.cc:482] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:09:07.590060 12942 raft_consensus.cc:3037] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:07.591665 12942 raft_consensus.cc:504] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 33021 } } peers { permanent_uuid: "daade419eb9f498994ac15c2bfa4911b" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45725 } } peers { permanent_uuid: "b728814d629c46d2ac07d56b1dff1e6e" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39363 } }
I20260620 20:09:07.592140 12942 leader_election.cc:288] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213 [CANDIDATE]: Term 1 election: Requested vote from peers daade419eb9f498994ac15c2bfa4911b (127.5.91.130:45725), b728814d629c46d2ac07d56b1dff1e6e (127.5.91.131:39363)
I20260620 20:09:07.592664 12657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c110cc0877ab41aeb3540c6d04835a69" candidate_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "daade419eb9f498994ac15c2bfa4911b"
I20260620 20:09:07.592736 12795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c110cc0877ab41aeb3540c6d04835a69" candidate_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b728814d629c46d2ac07d56b1dff1e6e"
I20260620 20:09:07.592793 12657 raft_consensus.cc:3037] T c110cc0877ab41aeb3540c6d04835a69 P daade419eb9f498994ac15c2bfa4911b [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:07.592927 12795 raft_consensus.cc:3037] T c110cc0877ab41aeb3540c6d04835a69 P b728814d629c46d2ac07d56b1dff1e6e [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:07.594030 12657 raft_consensus.cc:2454] T c110cc0877ab41aeb3540c6d04835a69 P daade419eb9f498994ac15c2bfa4911b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75c3d02cf5a54f0f8dbf93edc270a213 in term 1.
I20260620 20:09:07.594034 12795 raft_consensus.cc:2454] T c110cc0877ab41aeb3540c6d04835a69 P b728814d629c46d2ac07d56b1dff1e6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75c3d02cf5a54f0f8dbf93edc270a213 in term 1.
I20260620 20:09:07.594352 12456 leader_election.cc:302] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75c3d02cf5a54f0f8dbf93edc270a213, daade419eb9f498994ac15c2bfa4911b; no voters:
I20260620 20:09:07.594542 12942 raft_consensus.cc:2781] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:09:07.594762 12942 raft_consensus.cc:686] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213 [term 1 LEADER]: Becoming Leader. State: Replica: 75c3d02cf5a54f0f8dbf93edc270a213, State: Running, Role: LEADER
I20260620 20:09:07.595022 12942 consensus_queue.cc:237] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 33021 } } peers { permanent_uuid: "daade419eb9f498994ac15c2bfa4911b" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45725 } } peers { permanent_uuid: "b728814d629c46d2ac07d56b1dff1e6e" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39363 } }
I20260620 20:09:07.595494 12943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.595234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.5.91.129:53695)
0620 20:09:07.595345 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.595412 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260620 20:09:07.597569 12386 catalog_manager.cc:5495] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75c3d02cf5a54f0f8dbf93edc270a213 (127.5.91.129). New cstate: current_term: 1 leader_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c3d02cf5a54f0f8dbf93edc270a213" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 33021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "daade419eb9f498994ac15c2bfa4911b" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b728814d629c46d2ac07d56b1dff1e6e" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39363 } health_report { overall_health: UNKNOWN } } }
I20260620 20:09:07.627234 12928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.623909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50396 (local address 127.5.91.129:33021)
0620 20:09:07.624039 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:07.624043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:07.624057 (+ 14us) server_negotiation.cc:407] Connection header received
0620 20:09:07.624151 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:07.624154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:07.624208 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:07.624291 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:09:07.624535 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.625058 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.625880 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:07.626086 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:07.626187 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:07.626423 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:09:07.626859 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:09:07.626955 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:07.627043 (+ 88us) server_negotiation.cc:299] Negotiation successful
0620 20:09:07.627100 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 20:09:07.630515 12795 raft_consensus.cc:1261] T c110cc0877ab41aeb3540c6d04835a69 P b728814d629c46d2ac07d56b1dff1e6e [term 1 FOLLOWER]: Refusing update from remote peer 75c3d02cf5a54f0f8dbf93edc270a213: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:09:07.630970 12657 raft_consensus.cc:1261] T c110cc0877ab41aeb3540c6d04835a69 P daade419eb9f498994ac15c2bfa4911b [term 1 FOLLOWER]: Refusing update from remote peer 75c3d02cf5a54f0f8dbf93edc270a213: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:09:07.631129 12945 consensus_queue.cc:1035] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b728814d629c46d2ac07d56b1dff1e6e" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 39363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:09:07.631608 12942 consensus_queue.cc:1035] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213 [LEADER]: Connected to new peer: Peer: permanent_uuid: "daade419eb9f498994ac15c2bfa4911b" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:09:07.633411 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.632816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.5.91.131:41997)
0620 20:09:07.633259 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.633323 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365}
I20260620 20:09:07.634349 12941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.633682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.5.91.130:43661)
0620 20:09:07.634205 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.634263 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429}
I20260620 20:09:07.637022 12943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.636251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.5.91.129:49405)
0620 20:09:07.636890 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.636947 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":571}
I20260620 20:09:07.638767 12943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.638529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.5.91.129:43161)
0620 20:09:07.638655 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.638704 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260620 20:09:07.639355 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.639084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.5.91.131:45825)
0620 20:09:07.639190 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.639243 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260620 20:09:07.642757 12941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.642493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41567 (local address 127.5.91.130:37231)
0620 20:09:07.642624 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.642683 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260620 20:09:07.645015 12952 mvcc.cc:205] Tried to move back new op lower bound from 7299015260689448960 to 7299015260552036352. Current Snapshot: MvccSnapshot[applied={T|T < 7299015260689448960}]
I20260620 20:09:07.646062 12386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:09:07.646270 12386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:09:07.650864 12386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9b6482bf99b04fb6a6efbfa5ccf5b170: Sending DeleteTablet for 3 replicas of tablet c110cc0877ab41aeb3540c6d04835a69
I20260620 20:09:07.651624 12500 tablet_service.cc:1514] Processing DeleteTablet for tablet c110cc0877ab41aeb3540c6d04835a69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:09:07 UTC) from {username='oryx', principal='oryx/127.5.91.188@KRBTEST.COM'} at 127.0.0.1:50382
I20260620 20:09:07.651633 12637 tablet_service.cc:1514] Processing DeleteTablet for tablet c110cc0877ab41aeb3540c6d04835a69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:09:07 UTC) from {username='oryx', principal='oryx/127.5.91.188@KRBTEST.COM'} at 127.0.0.1:36892
I20260620 20:09:07.651815 12775 tablet_service.cc:1514] Processing DeleteTablet for tablet c110cc0877ab41aeb3540c6d04835a69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:09:07 UTC) from {username='oryx', principal='oryx/127.5.91.188@KRBTEST.COM'} at 127.0.0.1:52558
I20260620 20:09:07.652163 12961 tablet_replica.cc:330] T c110cc0877ab41aeb3540c6d04835a69 P daade419eb9f498994ac15c2bfa4911b: stopping tablet replica
I20260620 20:09:07.652297 12961 raft_consensus.cc:2229] T c110cc0877ab41aeb3540c6d04835a69 P daade419eb9f498994ac15c2bfa4911b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:09:07.652446 12961 raft_consensus.cc:2258] T c110cc0877ab41aeb3540c6d04835a69 P daade419eb9f498994ac15c2bfa4911b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:09:07.652593 12960 tablet_replica.cc:330] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213: stopping tablet replica
I20260620 20:09:07.652722 12960 raft_consensus.cc:2229] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213 [term 1 LEADER]: Raft consensus shutting down.
I20260620 20:09:07.652935 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 12340
I20260620 20:09:07.653038 12960 raft_consensus.cc:2258] T c110cc0877ab41aeb3540c6d04835a69 P 75c3d02cf5a54f0f8dbf93edc270a213 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:09:07.653893 12961 ts_tablet_manager.cc:1905] T c110cc0877ab41aeb3540c6d04835a69 P daade419eb9f498994ac15c2bfa4911b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:09:07.656141 12961 ts_tablet_manager.cc:1918] T c110cc0877ab41aeb3540c6d04835a69 P daade419eb9f498994ac15c2bfa4911b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:09:07.656214 12961 log.cc:1196] T c110cc0877ab41aeb3540c6d04835a69 P daade419eb9f498994ac15c2bfa4911b: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-1/wal/wals/c110cc0877ab41aeb3540c6d04835a69
I20260620 20:09:07.656486 12961 ts_tablet_manager.cc:1939] T c110cc0877ab41aeb3540c6d04835a69 P daade419eb9f498994ac15c2bfa4911b: Deleting consensus metadata
I20260620 20:09:07.657178 12374 catalog_manager.cc:4841] TS daade419eb9f498994ac15c2bfa4911b (127.5.91.130:45725): tablet c110cc0877ab41aeb3540c6d04835a69 (table test-table [id=aaac5b22bebf45f3b50373de884a395d]) successfully deleted
I20260620 20:09:07.658339 12962 tablet_replica.cc:330] T c110cc0877ab41aeb3540c6d04835a69 P b728814d629c46d2ac07d56b1dff1e6e: stopping tablet replica
I20260620 20:09:07.658494 12962 raft_consensus.cc:2229] T c110cc0877ab41aeb3540c6d04835a69 P b728814d629c46d2ac07d56b1dff1e6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:09:07.658638 12962 raft_consensus.cc:2258] T c110cc0877ab41aeb3540c6d04835a69 P b728814d629c46d2ac07d56b1dff1e6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:09:07.659232 12962 ts_tablet_manager.cc:1905] T c110cc0877ab41aeb3540c6d04835a69 P b728814d629c46d2ac07d56b1dff1e6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:09:07.660252 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 12574
I20260620 20:09:07.662005 12913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:07.661483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:33021 (local address 127.0.0.1:50398)
0620 20:09:07.661797 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:07.661897 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.129:33021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
W20260620 20:09:07.662385 12371 catalog_manager.cc:4584] TS 75c3d02cf5a54f0f8dbf93edc270a213 (127.5.91.129:33021): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c110cc0877ab41aeb3540c6d04835a69: Network error: Client connection negotiation failed: client connection to 127.5.91.129:33021: connect: Connection refused (error 111)
I20260620 20:09:07.663913 12962 ts_tablet_manager.cc:1918] T c110cc0877ab41aeb3540c6d04835a69 P b728814d629c46d2ac07d56b1dff1e6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:09:07.664003 12962 log.cc:1196] T c110cc0877ab41aeb3540c6d04835a69 P b728814d629c46d2ac07d56b1dff1e6e: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/c110cc0877ab41aeb3540c6d04835a69
I20260620 20:09:07.664253 12962 ts_tablet_manager.cc:1939] T c110cc0877ab41aeb3540c6d04835a69 P b728814d629c46d2ac07d56b1dff1e6e: Deleting consensus metadata
I20260620 20:09:07.665004 12374 catalog_manager.cc:4841] TS b728814d629c46d2ac07d56b1dff1e6e (127.5.91.131:39363): tablet c110cc0877ab41aeb3540c6d04835a69 (table test-table [id=aaac5b22bebf45f3b50373de884a395d]) successfully deleted
I20260620 20:09:07.668067 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 12710
I20260620 20:09:07.673391 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 12060
I20260620 20:09:07.678155 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 12163
2026-06-20T20:09:07Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6098 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:09:07 dist-test-slave-l8h4 krb5kdc[12965](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:09:07 dist-test-slave-l8h4 krb5kdc[12965](info): set up 2 sockets
Jun 20 20:09:07 dist-test-slave-l8h4 krb5kdc[12965](info): commencing operation
krb5kdc: starting...
W20260620 20:09:08.676225 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.974s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:09:08 dist-test-slave-l8h4 krb5kdc[12965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:09:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:09:08Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.190.keytab.
Entry for principal oryx/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.190.keytab.
I20260620 20:09:08.826910 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:36905
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.190.keytab
--principal=oryx/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:36905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:09:08.941216 12978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:08.941506 12978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:08.941548 12978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:08.945892 12978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:09:08.945981 12978 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 20:09:08.945998 12978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:08.946017 12978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:08.946031 12978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:09:08.946059 12978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:09:08.951889 12978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:36905
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.190.keytab
--principal=oryx/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:36905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:08.953593 12978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:08.954699 12978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:08.964207 12984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:08.964195 12986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:08.964148 12983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:08.964627 12978 server_base.cc:1034] running on GCE node
I20260620 20:09:08.965247 12978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:08.966507 12978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:08.969926 12978 hybrid_clock.cc:648] HybridClock initialized: now 1781986148969822 us; error 144 us; skew 500 ppm
Jun 20 20:09:08 dist-test-slave-l8h4 krb5kdc[12965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986148, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:08.979519 12978 init.cc:377] Logged in from keytab as oryx/127.5.91.190@KRBTEST.COM (short username oryx)
I20260620 20:09:08.980865 12978 webserver.cc:460] Webserver started at http://127.5.91.190:34763/ using document root <none> and password file <none>
I20260620 20:09:08.981603 12978 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:08.981684 12978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:08.981913 12978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:08.984372 12978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "89e55a1f00e44812af32fcc47b938170"
format_stamp: "Formatted at 2026-06-20 20:09:08 on dist-test-slave-l8h4"
server_key: "6419a46993838524e0204d0f719c8f51"
server_key_iv: "0db0c8f0088962f8e9382460daf7602e"
server_key_version: "encryptionkey@0"
I20260620 20:09:08.984972 12978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "89e55a1f00e44812af32fcc47b938170"
format_stamp: "Formatted at 2026-06-20 20:09:08 on dist-test-slave-l8h4"
server_key: "6419a46993838524e0204d0f719c8f51"
server_key_iv: "0db0c8f0088962f8e9382460daf7602e"
server_key_version: "encryptionkey@0"
I20260620 20:09:08.989784 12978 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260620 20:09:08.993623 12993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:08.995496 12978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:09:08.995626 12978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "89e55a1f00e44812af32fcc47b938170"
format_stamp: "Formatted at 2026-06-20 20:09:08 on dist-test-slave-l8h4"
server_key: "6419a46993838524e0204d0f719c8f51"
server_key_iv: "0db0c8f0088962f8e9382460daf7602e"
server_key_version: "encryptionkey@0"
I20260620 20:09:08.995767 12978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:09.013689 12978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:09.018616 12978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:09.019042 12978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:09.029111 12978 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:36905
I20260620 20:09:09.029080 13055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:36905 every 8 connection(s)
I20260620 20:09:09.030094 12978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:09:09.033907 13056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:09.034649 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 12978
I20260620 20:09:09.034798 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:09:09.035158 5486 external_mini_cluster.cc:1428] Setting key 4e338e43b9a9af0eca0a67255bb6a57b
I20260620 20:09:09.040271 13056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170: Bootstrap starting.
I20260620 20:09:09.042980 13056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:09.043891 13056 log.cc:824] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170: Log is configured to *not* fsync() on all Append() calls
Jun 20 20:09:09 dist-test-slave-l8h4 krb5kdc[12965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM
I20260620 20:09:09.046216 13056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170: No bootstrap required, opened a new log
I20260620 20:09:09.049930 13056 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e55a1f00e44812af32fcc47b938170" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 36905 } }
I20260620 20:09:09.050284 13056 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:09.050365 13056 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89e55a1f00e44812af32fcc47b938170, State: Initialized, Role: FOLLOWER
I20260620 20:09:09.050784 13056 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e55a1f00e44812af32fcc47b938170" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 36905 } }
I20260620 20:09:09.050923 13056 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:09:09.050987 13056 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:09:09.051105 13056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:09.051448 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:09.036864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51002 (local address 127.5.91.190:36905)
0620 20:09:09.037288 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:09.037305 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:09.037354 (+ 49us) server_negotiation.cc:407] Connection header received
0620 20:09:09.037988 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:09.038004 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:09.038376 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:09.038648 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:09.039530 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.040601 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.041712 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.042245 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.042466 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:09.045359 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:09.045365 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:09.045379 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:09.045408 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:09.047573 (+ 2165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:09.048174 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:09.048179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:09.048184 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:09.048301 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:09.048871 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:09.048877 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:09.048878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:09.049569 (+ 691us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:09.049787 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:09.050105 (+ 318us) server_negotiation.cc:299] Negotiation successful
0620 20:09:09.050549 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":132,"threads_started":1}
I20260620 20:09:09.052264 13056 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e55a1f00e44812af32fcc47b938170" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 36905 } }
I20260620 20:09:09.052619 13056 leader_election.cc:302] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89e55a1f00e44812af32fcc47b938170; no voters:
I20260620 20:09:09.052906 13056 leader_election.cc:288] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:09:09.053018 13061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:09:09.053254 13061 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170 [term 1 LEADER]: Becoming Leader. State: Replica: 89e55a1f00e44812af32fcc47b938170, State: Running, Role: LEADER
I20260620 20:09:09.053563 13061 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e55a1f00e44812af32fcc47b938170" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 36905 } }
I20260620 20:09:09.054569 13056 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:09:09.055891 13062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89e55a1f00e44812af32fcc47b938170" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e55a1f00e44812af32fcc47b938170" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 36905 } } }
I20260620 20:09:09.056057 13062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170 [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:09.055837 13063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89e55a1f00e44812af32fcc47b938170. Latest consensus state: current_term: 1 leader_uuid: "89e55a1f00e44812af32fcc47b938170" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e55a1f00e44812af32fcc47b938170" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 36905 } } }
I20260620 20:09:09.056334 13063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170 [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:09.057410 13070 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:09:09.060647 13070 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:09:09.068300 13070 catalog_manager.cc:1347] Generated new cluster ID: d04e8c4693da43488bb84a83d36d8dbc
I20260620 20:09:09.068396 13070 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:09:09.086822 13070 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:09:09.088207 13070 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:09:09.094521 13070 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170: Generated new TSK 0
I20260620 20:09:09.095340 13070 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.91.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.91.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.91.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.129.keytab.
Entry for principal oryx/127.5.91.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.129.keytab.
I20260620 20:09:09.135033 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:36905
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.129.keytab
--principal=oryx/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:09:09.257826 13082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:09.258116 13082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:09.258425 13082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:09.264163 13082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:09.264257 13082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:09.264307 13082 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 20:09:09.264345 13082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:09:09.270238 13082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.129.keytab
--principal=oryx/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.91.190:36905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:09.272125 13082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:09.273244 13082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:09.286042 13088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:09.286230 13090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:09.286078 13087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:09.286294 13082 server_base.cc:1034] running on GCE node
I20260620 20:09:09.286844 13082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:09.287711 13082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:09.288967 13082 hybrid_clock.cc:648] HybridClock initialized: now 1781986149288944 us; error 33 us; skew 500 ppm
Jun 20 20:09:09 dist-test-slave-l8h4 krb5kdc[12965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986149, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:09.292430 13082 init.cc:377] Logged in from keytab as oryx/127.5.91.129@KRBTEST.COM (short username oryx)
I20260620 20:09:09.293401 13082 webserver.cc:460] Webserver started at http://127.5.91.129:40067/ using document root <none> and password file <none>
I20260620 20:09:09.293954 13082 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:09.294021 13082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:09.294245 13082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:09.296336 13082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d"
format_stamp: "Formatted at 2026-06-20 20:09:09 on dist-test-slave-l8h4"
server_key: "3bb42b3fdac31c9b05fb40eb9caefeae"
server_key_iv: "a53df6230fb85fa3e81873df496e2517"
server_key_version: "encryptionkey@0"
I20260620 20:09:09.297150 13082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d"
format_stamp: "Formatted at 2026-06-20 20:09:09 on dist-test-slave-l8h4"
server_key: "3bb42b3fdac31c9b05fb40eb9caefeae"
server_key_iv: "a53df6230fb85fa3e81873df496e2517"
server_key_version: "encryptionkey@0"
I20260620 20:09:09.301826 13082 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260620 20:09:09.305357 13097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:09.307081 13082 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260620 20:09:09.307215 13082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d"
format_stamp: "Formatted at 2026-06-20 20:09:09 on dist-test-slave-l8h4"
server_key: "3bb42b3fdac31c9b05fb40eb9caefeae"
server_key_iv: "a53df6230fb85fa3e81873df496e2517"
server_key_version: "encryptionkey@0"
I20260620 20:09:09.307334 13082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:09.325927 13082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:09.329627 13082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:09.329838 13082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:09.331499 13082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:09.331563 13082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:09.331621 13082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:09.331651 13082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:09.342818 13082 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:43491
I20260620 20:09:09.343156 13212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:43491 every 8 connection(s)
Jun 20 20:09:09 dist-test-slave-l8h4 krb5kdc[12965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986149, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM
I20260620 20:09:09.343999 13082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:09:09.349069 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:09.332551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:43089 (local address 127.5.91.190:36905)
0620 20:09:09.332720 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:09.332723 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:09.333519 (+ 796us) server_negotiation.cc:407] Connection header received
0620 20:09:09.334665 (+ 1146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:09.334669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:09.334731 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:09.334788 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:09.336627 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.337405 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.339168 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.339592 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.339982 (+ 390us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:09.344112 (+ 4130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:09.344116 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:09.344119 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:09.344155 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:09.345684 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:09.346369 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:09.346372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:09.346374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:09.346510 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:09.347150 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:09.347154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:09.347156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:09.347367 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:09.347517 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:09.348726 (+ 1209us) server_negotiation.cc:299] Negotiation successful
0620 20:09:09.348899 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260620 20:09:09.349983 13106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:09.332823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:36905 (local address 127.5.91.129:43089)
0620 20:09:09.333343 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:09.333394 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:09.334427 (+ 1033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:09.335291 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:09.335300 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:09.335734 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:09.336486 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:09.336506 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.337568 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.337572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:09.339043 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:09.339056 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.340274 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.340278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:09.340577 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:09.341522 (+ 945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:09.341547 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:09.343908 (+ 2361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:09.345820 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:09.345826 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:09.345840 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:09.346127 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:09.346651 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:09.346655 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:09.346657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:09.346913 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:09.347516 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:09.347523 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:09.348188 (+ 665us) client_negotiation.cc:769] Sending connection context
0620 20:09:09.348512 (+ 324us) client_negotiation.cc:240] Negotiation successful
0620 20:09:09.349067 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":126,"threads_started":1}
I20260620 20:09:09.351475 13213 heartbeater.cc:347] Connected to a master server at 127.5.91.190:36905
I20260620 20:09:09.351637 13213 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:09.352319 13213 heartbeater.cc:511] Master 127.5.91.190:36905 requested a full tablet report, sending...
I20260620 20:09:09.353204 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 13082
I20260620 20:09:09.353338 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:09:09.353623 5486 external_mini_cluster.cc:1428] Setting key 119e0115f0e936b12fd16ac1b684d484
I20260620 20:09:09.354214 13009 ts_manager.cc:194] Registered new tserver with Master: 2402c8bc50de4c2e93d2fa5e8ebc3a2d (127.5.91.129:43491)
I20260620 20:09:09.355775 13009 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:43089
I20260620 20:09:09.363790 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:09.354111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:32985 (local address 127.5.91.190:36905)
0620 20:09:09.354251 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:09.354256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:09.355028 (+ 772us) server_negotiation.cc:407] Connection header received
0620 20:09:09.355196 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:09.355199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:09.355249 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:09.355332 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:09.356148 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.356926 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.357765 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.357975 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.358060 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:09.359201 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:09.359205 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:09.359207 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:09.359232 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:09.361175 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:09.361724 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:09.361731 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:09.361733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:09.361917 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:09.362583 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:09.362587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:09.362589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:09.362855 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:09.362988 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:09.363476 (+ 488us) server_negotiation.cc:299] Negotiation successful
0620 20:09:09.363612 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260620 20:09:09.364012 13219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:09.354048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:36905 (local address 127.5.91.129:32985)
0620 20:09:09.354941 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:09.354956 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:09.355061 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:09.355356 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:09.355359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:09.355717 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:09.355949 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:09.355962 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.357069 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.357071 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:09.357655 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:09.357664 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.358076 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.358080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:09.358181 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:09.358708 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:09.358720 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:09.359062 (+ 342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:09.361308 (+ 2246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:09.361311 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:09.361312 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:09.361520 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:09.362302 (+ 782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:09.362306 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:09.362308 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:09.362432 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:09.363061 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:09.363064 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:09.363211 (+ 147us) client_negotiation.cc:769] Sending connection context
0620 20:09:09.363360 (+ 149us) client_negotiation.cc:240] Negotiation successful
0620 20:09:09.363556 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"thread_start_us":141,"threads_started":1}
WARNING: no policy specified for oryx/127.5.91.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.91.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.91.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.130.keytab.
Entry for principal oryx/127.5.91.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.130.keytab.
I20260620 20:09:09.388486 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:36905
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.130.keytab
--principal=oryx/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:09:09.503669 13222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:09.503949 13222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:09.503994 13222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:09.508093 13222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:09.508162 13222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:09.508203 13222 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 20:09:09.508242 13222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:09:09.513759 13222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.130.keytab
--principal=oryx/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.91.190:36905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:09.514922 13222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:09.515899 13222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 20:09:09.525544 13222 server_base.cc:1034] running on GCE node
W20260620 20:09:09.525463 13227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:09.525590 13228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:09.525488 13230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:09.526083 13222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:09.526734 13222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:09.527911 13222 hybrid_clock.cc:648] HybridClock initialized: now 1781986149527882 us; error 44 us; skew 500 ppm
Jun 20 20:09:09 dist-test-slave-l8h4 krb5kdc[12965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986149, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:09.530697 13222 init.cc:377] Logged in from keytab as oryx/127.5.91.130@KRBTEST.COM (short username oryx)
I20260620 20:09:09.531559 13222 webserver.cc:460] Webserver started at http://127.5.91.130:33483/ using document root <none> and password file <none>
I20260620 20:09:09.532080 13222 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:09.532141 13222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:09.532294 13222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:09.534296 13222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "c8a411d4fd0d48df8446d518e2d71a77"
format_stamp: "Formatted at 2026-06-20 20:09:09 on dist-test-slave-l8h4"
server_key: "3e33a250ea02a34148334e7d1807c5e8"
server_key_iv: "cf51ae24db65d4a1b8516d07b40c99bf"
server_key_version: "encryptionkey@0"
I20260620 20:09:09.534830 13222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "c8a411d4fd0d48df8446d518e2d71a77"
format_stamp: "Formatted at 2026-06-20 20:09:09 on dist-test-slave-l8h4"
server_key: "3e33a250ea02a34148334e7d1807c5e8"
server_key_iv: "cf51ae24db65d4a1b8516d07b40c99bf"
server_key_version: "encryptionkey@0"
I20260620 20:09:09.538784 13222 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260620 20:09:09.541200 13237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:09.542435 13222 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260620 20:09:09.542538 13222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "c8a411d4fd0d48df8446d518e2d71a77"
format_stamp: "Formatted at 2026-06-20 20:09:09 on dist-test-slave-l8h4"
server_key: "3e33a250ea02a34148334e7d1807c5e8"
server_key_iv: "cf51ae24db65d4a1b8516d07b40c99bf"
server_key_version: "encryptionkey@0"
I20260620 20:09:09.542644 13222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:09.558728 13222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:09.561981 13222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:09.562179 13222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:09.563679 13222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:09.563738 13222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:09.563794 13222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:09.563822 13222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:09.574365 13222 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:40845
I20260620 20:09:09.574504 13352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:40845 every 8 connection(s)
Jun 20 20:09:09 dist-test-slave-l8h4 krb5kdc[12965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986149, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM
I20260620 20:09:09.575394 13222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:09:09.579680 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:09.565410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:56441 (local address 127.5.91.190:36905)
0620 20:09:09.565538 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:09.565542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:09.566867 (+ 1325us) server_negotiation.cc:407] Connection header received
0620 20:09:09.568108 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:09.568112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:09.568167 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:09.568256 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:09.569819 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.570654 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.571949 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.572268 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.572381 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:09.575312 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:09.575316 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:09.575319 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:09.575348 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:09.577082 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:09.577606 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:09.577609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:09.577610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:09.577694 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:09.578470 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:09.578474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:09.578476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:09.578703 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:09.578818 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:09.579369 (+ 551us) server_negotiation.cc:299] Negotiation successful
0620 20:09:09.579502 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 20:09:09.580442 13246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:09.566075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:36905 (local address 127.5.91.130:56441)
0620 20:09:09.566696 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:09.566754 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:09.567844 (+ 1090us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:09.568450 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:09.568460 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:09.568954 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:09.569644 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:09.569665 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.570791 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.570795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:09.571823 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:09.571833 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.572423 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.572428 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:09.572642 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:09.573259 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:09.573280 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:09.575092 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:09.577199 (+ 2107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:09.577204 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:09.577214 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:09.577477 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:09.577928 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:09.577932 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:09.577934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:09.578330 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:09.578853 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:09.578860 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:09.579114 (+ 254us) client_negotiation.cc:769] Sending connection context
0620 20:09:09.579289 (+ 175us) client_negotiation.cc:240] Negotiation successful
0620 20:09:09.579622 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":142,"threads_started":1}
I20260620 20:09:09.581735 13353 heartbeater.cc:347] Connected to a master server at 127.5.91.190:36905
I20260620 20:09:09.581873 13353 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:09.582377 13353 heartbeater.cc:511] Master 127.5.91.190:36905 requested a full tablet report, sending...
I20260620 20:09:09.583371 13009 ts_manager.cc:194] Registered new tserver with Master: c8a411d4fd0d48df8446d518e2d71a77 (127.5.91.130:40845)
I20260620 20:09:09.584005 13009 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:56441
I20260620 20:09:09.585605 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 13222
I20260620 20:09:09.585696 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:09:09.585937 5486 external_mini_cluster.cc:1428] Setting key 1419887ac028896b62196457322defc2
I20260620 20:09:09.591908 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:09.583991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:57685 (local address 127.5.91.190:36905)
0620 20:09:09.584105 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:09.584109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:09.584220 (+ 111us) server_negotiation.cc:407] Connection header received
0620 20:09:09.584336 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:09.584338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:09.584377 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:09.584436 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:09.584837 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.585328 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.587038 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.587324 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.587428 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:09.588628 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:09.588632 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:09.588635 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:09.588661 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:09.590044 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:09.590550 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:09.590553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:09.590560 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:09.590638 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:09.590965 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:09.590968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:09.590969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:09.591192 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:09.591278 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:09.591655 (+ 377us) server_negotiation.cc:299] Negotiation successful
0620 20:09:09.591751 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 20:09:09.591950 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:09.583794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:36905 (local address 127.5.91.130:57685)
0620 20:09:09.584137 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:09.584151 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:09.584252 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:09.584462 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:09.584464 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:09.584644 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:09.584737 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:09.584742 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.586267 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.586270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:09.586920 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:09.586931 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.587474 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.587476 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:09.587548 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:09.588141 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:09.588154 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:09.588492 (+ 338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:09.590181 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:09.590184 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:09.590187 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:09.590433 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:09.590739 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:09.590742 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:09.590744 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:09.590847 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:09.591293 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:09.591297 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:09.591426 (+ 129us) client_negotiation.cc:769] Sending connection context
0620 20:09:09.591550 (+ 124us) client_negotiation.cc:240] Negotiation successful
0620 20:09:09.591722 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":112,"threads_started":1}
WARNING: no policy specified for oryx/127.5.91.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.91.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.91.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.131.keytab.
Entry for principal oryx/127.5.91.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.131.keytab.
I20260620 20:09:09.617769 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:36905
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.131.keytab
--principal=oryx/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:09:09.738834 13362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:09.739144 13362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:09.739190 13362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:09.743315 13362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:09.743407 13362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:09.743462 13362 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 20:09:09.743499 13362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:09:09.749285 13362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.131.keytab
--principal=oryx/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.91.190:36905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:09.750624 13362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:09.751736 13362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:09.762746 13367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:09.762737 13368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:09.762737 13370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:09.763123 13362 server_base.cc:1034] running on GCE node
I20260620 20:09:09.763623 13362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:09.764847 13362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:09.766062 13362 hybrid_clock.cc:648] HybridClock initialized: now 1781986149766036 us; error 40 us; skew 500 ppm
Jun 20 20:09:09 dist-test-slave-l8h4 krb5kdc[12965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986149, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:09.768841 13362 init.cc:377] Logged in from keytab as oryx/127.5.91.131@KRBTEST.COM (short username oryx)
I20260620 20:09:09.769838 13362 webserver.cc:460] Webserver started at http://127.5.91.131:43161/ using document root <none> and password file <none>
I20260620 20:09:09.770633 13362 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:09.770706 13362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:09.770903 13362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:09.773018 13362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "bceef69fdc22452ca520d306ac58656f"
format_stamp: "Formatted at 2026-06-20 20:09:09 on dist-test-slave-l8h4"
server_key: "ca68ec195d59a3328ca017bc2de19f6c"
server_key_iv: "59566fe565e0ec5c38051f6fd0b15e2d"
server_key_version: "encryptionkey@0"
I20260620 20:09:09.773561 13362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "bceef69fdc22452ca520d306ac58656f"
format_stamp: "Formatted at 2026-06-20 20:09:09 on dist-test-slave-l8h4"
server_key: "ca68ec195d59a3328ca017bc2de19f6c"
server_key_iv: "59566fe565e0ec5c38051f6fd0b15e2d"
server_key_version: "encryptionkey@0"
I20260620 20:09:09.778421 13362 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260620 20:09:09.782920 13377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:09.784603 13362 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260620 20:09:09.784732 13362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "bceef69fdc22452ca520d306ac58656f"
format_stamp: "Formatted at 2026-06-20 20:09:09 on dist-test-slave-l8h4"
server_key: "ca68ec195d59a3328ca017bc2de19f6c"
server_key_iv: "59566fe565e0ec5c38051f6fd0b15e2d"
server_key_version: "encryptionkey@0"
I20260620 20:09:09.784864 13362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:09.817556 13362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:09.820658 13362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:09.820845 13362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:09.822333 13362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:09.822388 13362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:09.822443 13362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:09.822468 13362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 20:09:09 dist-test-slave-l8h4 krb5kdc[12965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986149, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM
I20260620 20:09:09.833168 13362 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:33291
I20260620 20:09:09.833201 13492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:33291 every 8 connection(s)
I20260620 20:09:09.834156 13362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:09:09.835136 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 13362
I20260620 20:09:09.835338 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:09:09.835675 5486 external_mini_cluster.cc:1428] Setting key e042c63377738918a68a3d9607cbb546
I20260620 20:09:09.838965 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:09.823145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:42595 (local address 127.5.91.190:36905)
0620 20:09:09.823300 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:09.823305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:09.824126 (+ 821us) server_negotiation.cc:407] Connection header received
0620 20:09:09.825147 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:09.825151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:09.825209 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:09.825265 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:09.826955 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.827529 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.829105 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.829409 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.829520 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:09.833365 (+ 3845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:09.833370 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:09.833372 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:09.833405 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:09.835659 (+ 2254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:09.836910 (+ 1251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:09.836913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:09.836915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:09.837020 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:09.837465 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:09.837469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:09.837471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:09.837740 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:09.837841 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:09.838657 (+ 816us) server_negotiation.cc:299] Negotiation successful
0620 20:09:09.838808 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 20:09:09.839910 13386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:09.823509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:36905 (local address 127.5.91.131:42595)
0620 20:09:09.823958 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:09.824014 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:09.824901 (+ 887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:09.825680 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:09.825689 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:09.826092 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:09.826788 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:09.826810 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.827656 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.827659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:09.828966 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:09.828978 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.829581 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.829586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:09.829799 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:09.831038 (+ 1239us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:09.831060 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:09.833154 (+ 2094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:09.835893 (+ 2739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:09.835903 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:09.835924 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:09.836802 (+ 878us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:09.837128 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:09.837132 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:09.837134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:09.837342 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:09.837851 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:09.837857 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:09.838123 (+ 266us) client_negotiation.cc:769] Sending connection context
0620 20:09:09.838373 (+ 250us) client_negotiation.cc:240] Negotiation successful
0620 20:09:09.838839 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":101,"threads_started":1}
I20260620 20:09:09.842252 13493 heartbeater.cc:347] Connected to a master server at 127.5.91.190:36905
I20260620 20:09:09.842883 13493 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:09.844501 13493 heartbeater.cc:511] Master 127.5.91.190:36905 requested a full tablet report, sending...
I20260620 20:09:09.845680 13009 ts_manager.cc:194] Registered new tserver with Master: bceef69fdc22452ca520d306ac58656f (127.5.91.131:33291)
I20260620 20:09:09.846499 13009 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:42595
I20260620 20:09:09.850500 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 20:09:09.854852 13499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:09.844684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:36905 (local address 127.5.91.131:39297)
0620 20:09:09.845119 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:09.845134 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:09.845245 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:09.845539 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:09.845542 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:09.845832 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:09.845964 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:09.845972 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.846993 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.846997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:09.847959 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:09.847969 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.848535 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.848539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:09.848622 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:09.849269 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:09.849280 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:09.849945 (+ 665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:09.853370 (+ 3425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:09.853374 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:09.853376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:09.853587 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:09.853841 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:09.853844 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:09.853846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:09.853926 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:09.854340 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:09.854343 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:09.854452 (+ 109us) client_negotiation.cc:769] Sending connection context
0620 20:09:09.854558 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 20:09:09.854696 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":152,"threads_started":1}
I20260620 20:09:09.855134 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:09.845180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:39297 (local address 127.5.91.190:36905)
0620 20:09:09.845319 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:09.845323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:09.845335 (+ 12us) server_negotiation.cc:407] Connection header received
0620 20:09:09.845372 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:09.845375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:09.845420 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:09.845499 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:09.846101 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.846816 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.848084 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.848385 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.848495 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:09.850072 (+ 1577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:09.850075 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:09.850078 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:09.850103 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:09.853242 (+ 3139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:09.853680 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:09.853683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:09.853685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:09.853761 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:09.854006 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:09.854009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:09.854011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:09.854257 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:09.854340 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:09.854697 (+ 357us) server_negotiation.cc:299] Negotiation successful
0620 20:09:09.854824 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Jun 20 20:09:09 dist-test-slave-l8h4 krb5kdc[12965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986149, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 20:09:09 dist-test-slave-l8h4 krb5kdc[12965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:09 dist-test-slave-l8h4 krb5kdc[12965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
W20260620 20:09:09.874970 13059 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:09.868428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51012 (local address 127.5.91.190:36905)
0620 20:09:09.868624 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:09.868628 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:09.868644 (+ 16us) server_negotiation.cc:407] Connection header received
0620 20:09:09.868685 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:09.868688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:09.868729 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:09.868809 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:09.869619 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.870361 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.871457 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.871668 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.871755 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:09.874833 (+ 3078us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51012: BlockingRecv error: recv got EOF from 127.0.0.1:51012 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 20 20:09:09 dist-test-slave-l8h4 krb5kdc[12965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986149, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM
I20260620 20:09:09.884539 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:09.876505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51028 (local address 127.5.91.190:36905)
0620 20:09:09.876655 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:09.876659 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:09.876757 (+ 98us) server_negotiation.cc:407] Connection header received
0620 20:09:09.876936 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:09.876939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:09.876988 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:09.877054 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:09.877636 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.878127 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.878963 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.879201 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.879304 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:09.881580 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:09.881583 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:09.881585 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:09.881607 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:09.883038 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:09.883439 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:09.883443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:09.883445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:09.883521 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:09.883763 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:09.883766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:09.883768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:09.883950 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:09.884062 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:09.884279 (+ 217us) server_negotiation.cc:299] Negotiation successful
0620 20:09:09.884387 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260620 20:09:09.887630 13009 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51028:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:09:09.890892 13009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:09:09.904516 13515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:09.899342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:43491 (local address 127.0.0.1:49750)
0620 20:09:09.899677 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:09.899728 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:09.899871 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:09.900624 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:09.900630 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:09.900653 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:09.900795 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:09.900803 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.901988 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.901991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:09.902973 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:09.902984 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.903986 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.903990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:09.904101 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:09.904141 (+ 40us) client_negotiation.cc:769] Sending connection context
0620 20:09:09.904235 (+ 94us) client_negotiation.cc:240] Negotiation successful
0620 20:09:09.904358 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":84,"threads_started":1}
I20260620 20:09:09.904516 13516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:09.899499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49750 (local address 127.5.91.129:43491)
0620 20:09:09.900070 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:09.900078 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:09.900104 (+ 26us) server_negotiation.cc:407] Connection header received
0620 20:09:09.900179 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:09.900184 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:09.900434 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:09.900606 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:09.900923 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.901878 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.903094 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.903825 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.903933 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:09.904086 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:09.904281 (+ 195us) server_negotiation.cc:299] Negotiation successful
0620 20:09:09.904378 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":85,"threads_started":1}
I20260620 20:09:09.906015 13517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:09.899662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:33291 (local address 127.0.0.1:53252)
0620 20:09:09.900321 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:09.900334 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:09.900478 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:09.902094 (+ 1616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:09.902097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:09.902115 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:09.902282 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:09.902289 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.903683 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.903686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:09.904806 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:09.904817 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.905613 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.905617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:09.905707 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:09.905732 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 20:09:09.905797 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 20:09:09.905872 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"mutex_wait_us":55,"thread_start_us":403,"threads_started":1}
I20260620 20:09:09.906431 13519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:09.900139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53252 (local address 127.5.91.131:33291)
0620 20:09:09.901498 (+ 1359us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:09.901512 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:09.901531 (+ 19us) server_negotiation.cc:407] Connection header received
0620 20:09:09.901596 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:09.901600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:09.901757 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:09.901959 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:09.902407 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.903570 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.904946 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.905492 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.905591 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:09.905758 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:09.906055 (+ 297us) server_negotiation.cc:299] Negotiation successful
0620 20:09:09.906276 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1267,"thread_start_us":106,"threads_started":1}
I20260620 20:09:09.906865 13518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:09.900031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:40845 (local address 127.0.0.1:33930)
0620 20:09:09.901117 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:09.901131 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:09.901233 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:09.901726 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:09.901729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:09.901748 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:09.901832 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:09.901838 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.903487 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.903490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:09.904910 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:09.904918 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.906108 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.906112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:09.906338 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:09.906356 (+ 18us) client_negotiation.cc:769] Sending connection context
0620 20:09:09.906584 (+ 228us) client_negotiation.cc:240] Negotiation successful
0620 20:09:09.906742 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":962,"mutex_wait_us":28,"thread_start_us":85,"threads_started":1}
I20260620 20:09:09.907768 13520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:09.900498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33930 (local address 127.5.91.130:40845)
0620 20:09:09.900915 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:09.900934 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:09.901225 (+ 291us) server_negotiation.cc:407] Connection header received
0620 20:09:09.901390 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:09.901396 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:09.901579 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:09.901717 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:09.901956 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.903353 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.905022 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:09.905805 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:09.905893 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:09.906046 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:09.907509 (+ 1463us) server_negotiation.cc:299] Negotiation successful
0620 20:09:09.907625 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":91,"threads_started":1}
I20260620 20:09:09.908696 13147 tablet_service.cc:1467] Processing CreateTablet for tablet 95c2b842fb614b49beefc98dfab94837 (DEFAULT_TABLE table=test-table [id=484777c6c0a04ec3a110e09189475d87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:09:09.909657 13147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95c2b842fb614b49beefc98dfab94837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:09.909453 13427 tablet_service.cc:1467] Processing CreateTablet for tablet 95c2b842fb614b49beefc98dfab94837 (DEFAULT_TABLE table=test-table [id=484777c6c0a04ec3a110e09189475d87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:09:09.910481 13427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95c2b842fb614b49beefc98dfab94837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:09.910528 13287 tablet_service.cc:1467] Processing CreateTablet for tablet 95c2b842fb614b49beefc98dfab94837 (DEFAULT_TABLE table=test-table [id=484777c6c0a04ec3a110e09189475d87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:09:09.911469 13287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95c2b842fb614b49beefc98dfab94837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:09.917013 13521 tablet_bootstrap.cc:492] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Bootstrap starting.
I20260620 20:09:09.918457 13522 tablet_bootstrap.cc:492] T 95c2b842fb614b49beefc98dfab94837 P bceef69fdc22452ca520d306ac58656f: Bootstrap starting.
I20260620 20:09:09.919237 13521 tablet_bootstrap.cc:654] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:09.919397 13523 tablet_bootstrap.cc:492] T 95c2b842fb614b49beefc98dfab94837 P c8a411d4fd0d48df8446d518e2d71a77: Bootstrap starting.
I20260620 20:09:09.920243 13521 log.cc:824] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Log is configured to *not* fsync() on all Append() calls
I20260620 20:09:09.920548 13522 tablet_bootstrap.cc:654] T 95c2b842fb614b49beefc98dfab94837 P bceef69fdc22452ca520d306ac58656f: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:09.921110 13523 tablet_bootstrap.cc:654] T 95c2b842fb614b49beefc98dfab94837 P c8a411d4fd0d48df8446d518e2d71a77: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:09.921293 13522 log.cc:824] T 95c2b842fb614b49beefc98dfab94837 P bceef69fdc22452ca520d306ac58656f: Log is configured to *not* fsync() on all Append() calls
I20260620 20:09:09.921854 13523 log.cc:824] T 95c2b842fb614b49beefc98dfab94837 P c8a411d4fd0d48df8446d518e2d71a77: Log is configured to *not* fsync() on all Append() calls
I20260620 20:09:09.922405 13521 tablet_bootstrap.cc:492] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: No bootstrap required, opened a new log
I20260620 20:09:09.922565 13521 ts_tablet_manager.cc:1397] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260620 20:09:09.923341 13522 tablet_bootstrap.cc:492] T 95c2b842fb614b49beefc98dfab94837 P bceef69fdc22452ca520d306ac58656f: No bootstrap required, opened a new log
I20260620 20:09:09.923516 13522 ts_tablet_manager.cc:1397] T 95c2b842fb614b49beefc98dfab94837 P bceef69fdc22452ca520d306ac58656f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 20:09:09.924175 13523 tablet_bootstrap.cc:492] T 95c2b842fb614b49beefc98dfab94837 P c8a411d4fd0d48df8446d518e2d71a77: No bootstrap required, opened a new log
I20260620 20:09:09.924355 13523 ts_tablet_manager.cc:1397] T 95c2b842fb614b49beefc98dfab94837 P c8a411d4fd0d48df8446d518e2d71a77: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260620 20:09:09.926409 13522 raft_consensus.cc:348] T 95c2b842fb614b49beefc98dfab94837 P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:09.926644 13522 raft_consensus.cc:374] T 95c2b842fb614b49beefc98dfab94837 P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:09.926700 13522 raft_consensus.cc:729] T 95c2b842fb614b49beefc98dfab94837 P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bceef69fdc22452ca520d306ac58656f, State: Initialized, Role: FOLLOWER
I20260620 20:09:09.927179 13522 consensus_queue.cc:260] T 95c2b842fb614b49beefc98dfab94837 P bceef69fdc22452ca520d306ac58656f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:09.927274 13521 raft_consensus.cc:348] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:09.927541 13521 raft_consensus.cc:374] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:09.927600 13521 raft_consensus.cc:729] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2402c8bc50de4c2e93d2fa5e8ebc3a2d, State: Initialized, Role: FOLLOWER
I20260620 20:09:09.927980 13493 heartbeater.cc:503] Master 127.5.91.190:36905 was elected leader, sending a full tablet report...
I20260620 20:09:09.928046 13521 consensus_queue.cc:260] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:09.928328 13522 ts_tablet_manager.cc:1428] T 95c2b842fb614b49beefc98dfab94837 P bceef69fdc22452ca520d306ac58656f: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260620 20:09:09.928685 13213 heartbeater.cc:503] Master 127.5.91.190:36905 was elected leader, sending a full tablet report...
I20260620 20:09:09.928958 13521 ts_tablet_manager.cc:1428] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260620 20:09:09.930064 13523 raft_consensus.cc:348] T 95c2b842fb614b49beefc98dfab94837 P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:09.930347 13523 raft_consensus.cc:374] T 95c2b842fb614b49beefc98dfab94837 P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:09.930399 13523 raft_consensus.cc:729] T 95c2b842fb614b49beefc98dfab94837 P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8a411d4fd0d48df8446d518e2d71a77, State: Initialized, Role: FOLLOWER
I20260620 20:09:09.930835 13523 consensus_queue.cc:260] T 95c2b842fb614b49beefc98dfab94837 P c8a411d4fd0d48df8446d518e2d71a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:09.931491 13353 heartbeater.cc:503] Master 127.5.91.190:36905 was elected leader, sending a full tablet report...
I20260620 20:09:09.931857 13523 ts_tablet_manager.cc:1428] T 95c2b842fb614b49beefc98dfab94837 P c8a411d4fd0d48df8446d518e2d71a77: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260620 20:09:10.004559 13528 raft_consensus.cc:482] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:09:10.004951 13528 raft_consensus.cc:504] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.006656 13528 leader_election.cc:288] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8a411d4fd0d48df8446d518e2d71a77 (127.5.91.130:40845), bceef69fdc22452ca520d306ac58656f (127.5.91.131:33291)
I20260620 20:09:10.011682 13531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:10.007211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:40845 (local address 127.5.91.129:37861)
0620 20:09:10.007696 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:10.007715 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:10.007915 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:10.008156 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:10.008159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:10.008174 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:10.008289 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:10.008295 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.009285 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.009288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:10.010164 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:10.010174 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.011185 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.011189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:10.011321 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:10.011340 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 20:09:10.011412 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 20:09:10.011500 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"mutex_wait_us":24,"thread_start_us":177,"threads_started":1}
I20260620 20:09:10.011646 13520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:10.007371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:37861 (local address 127.5.91.130:40845)
0620 20:09:10.007623 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:10.007628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:10.007903 (+ 275us) server_negotiation.cc:407] Connection header received
0620 20:09:10.008020 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:10.008023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:10.008069 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:10.008131 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:10.008416 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.009112 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.010503 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.011049 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.011185 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:10.011243 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:10.011437 (+ 194us) server_negotiation.cc:299] Negotiation successful
0620 20:09:10.011510 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260620 20:09:10.011772 13519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:10.007086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:60423 (local address 127.5.91.131:33291)
0620 20:09:10.007231 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:10.007242 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:10.007479 (+ 237us) server_negotiation.cc:407] Connection header received
0620 20:09:10.007749 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:10.007753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:10.007798 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:10.007907 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:10.008167 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.008962 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.010406 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.011216 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.011344 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:10.011402 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:10.011588 (+ 186us) server_negotiation.cc:299] Negotiation successful
0620 20:09:10.011655 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 20:09:10.011847 13530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:10.006949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:33291 (local address 127.5.91.129:60423)
0620 20:09:10.007382 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:10.007395 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:10.007622 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:10.007923 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:10.007926 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:10.007944 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:10.008063 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:10.008068 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.009111 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.009115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:10.010126 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:10.010240 (+ 114us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.011344 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.011348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:10.011475 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:10.011496 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 20:09:10.011562 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 20:09:10.011638 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":27,"thread_start_us":135,"threads_started":1}
I20260620 20:09:10.012735 13307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95c2b842fb614b49beefc98dfab94837" candidate_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8a411d4fd0d48df8446d518e2d71a77" is_pre_election: true
I20260620 20:09:10.013075 13307 raft_consensus.cc:2454] T 95c2b842fb614b49beefc98dfab94837 P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2402c8bc50de4c2e93d2fa5e8ebc3a2d in term 0.
I20260620 20:09:10.013177 13447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95c2b842fb614b49beefc98dfab94837" candidate_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bceef69fdc22452ca520d306ac58656f" is_pre_election: true
I20260620 20:09:10.013428 13447 raft_consensus.cc:2454] T 95c2b842fb614b49beefc98dfab94837 P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2402c8bc50de4c2e93d2fa5e8ebc3a2d in term 0.
I20260620 20:09:10.013886 13101 leader_election.cc:302] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2402c8bc50de4c2e93d2fa5e8ebc3a2d, bceef69fdc22452ca520d306ac58656f; no voters:
I20260620 20:09:10.014241 13528 raft_consensus.cc:2781] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:09:10.014334 13528 raft_consensus.cc:482] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:09:10.014377 13528 raft_consensus.cc:3037] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:10.015452 13528 raft_consensus.cc:504] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.015945 13528 leader_election.cc:288] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [CANDIDATE]: Term 1 election: Requested vote from peers c8a411d4fd0d48df8446d518e2d71a77 (127.5.91.130:40845), bceef69fdc22452ca520d306ac58656f (127.5.91.131:33291)
I20260620 20:09:10.016705 13447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95c2b842fb614b49beefc98dfab94837" candidate_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bceef69fdc22452ca520d306ac58656f"
I20260620 20:09:10.016841 13447 raft_consensus.cc:3037] T 95c2b842fb614b49beefc98dfab94837 P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:10.016526 13307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95c2b842fb614b49beefc98dfab94837" candidate_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8a411d4fd0d48df8446d518e2d71a77"
I20260620 20:09:10.017088 13307 raft_consensus.cc:3037] T 95c2b842fb614b49beefc98dfab94837 P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:10.018925 13307 raft_consensus.cc:2454] T 95c2b842fb614b49beefc98dfab94837 P c8a411d4fd0d48df8446d518e2d71a77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2402c8bc50de4c2e93d2fa5e8ebc3a2d in term 1.
I20260620 20:09:10.019129 13447 raft_consensus.cc:2454] T 95c2b842fb614b49beefc98dfab94837 P bceef69fdc22452ca520d306ac58656f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2402c8bc50de4c2e93d2fa5e8ebc3a2d in term 1.
I20260620 20:09:10.019999 13098 leader_election.cc:302] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2402c8bc50de4c2e93d2fa5e8ebc3a2d, c8a411d4fd0d48df8446d518e2d71a77; no voters:
I20260620 20:09:10.020691 13528 raft_consensus.cc:2781] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:09:10.021147 13528 raft_consensus.cc:686] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 1 LEADER]: Becoming Leader. State: Replica: 2402c8bc50de4c2e93d2fa5e8ebc3a2d, State: Running, Role: LEADER
I20260620 20:09:10.021524 13528 consensus_queue.cc:237] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.026199 13010 catalog_manager.cc:5495] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2402c8bc50de4c2e93d2fa5e8ebc3a2d (127.5.91.129). New cstate: current_term: 1 leader_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } health_report { overall_health: UNKNOWN } } }
I20260620 20:09:10.064251 13516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:10.060344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49766 (local address 127.5.91.129:43491)
0620 20:09:10.060530 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:10.060534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:10.060552 (+ 18us) server_negotiation.cc:407] Connection header received
0620 20:09:10.060627 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:10.060629 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:10.060685 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:10.060788 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:09:10.061201 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.061712 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.062653 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.062872 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.063014 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:10.063268 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:09:10.063720 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:09:10.063804 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:10.064034 (+ 230us) server_negotiation.cc:299] Negotiation successful
0620 20:09:10.064101 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260620 20:09:10.070670 13447 raft_consensus.cc:1261] T 95c2b842fb614b49beefc98dfab94837 P bceef69fdc22452ca520d306ac58656f [term 1 FOLLOWER]: Refusing update from remote peer 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:09:10.070781 13307 raft_consensus.cc:1261] T 95c2b842fb614b49beefc98dfab94837 P c8a411d4fd0d48df8446d518e2d71a77 [term 1 FOLLOWER]: Refusing update from remote peer 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:09:10.071364 13528 consensus_queue.cc:1035] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:09:10.071590 13532 consensus_queue.cc:1035] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 20:09:10.076907 13354 tablet.cc:2369] T 95c2b842fb614b49beefc98dfab94837 P c8a411d4fd0d48df8446d518e2d71a77: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:09:10.090824 13009 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:09:10.091054 13009 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:09:10.093828 13009 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170: Sending DeleteTablet for 3 replicas of tablet 95c2b842fb614b49beefc98dfab94837
I20260620 20:09:10.095057 13427 tablet_service.cc:1514] Processing DeleteTablet for tablet 95c2b842fb614b49beefc98dfab94837 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:09:10 UTC) from {username='oryx', principal='oryx/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:53252
I20260620 20:09:10.095054 13147 tablet_service.cc:1514] Processing DeleteTablet for tablet 95c2b842fb614b49beefc98dfab94837 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:09:10 UTC) from {username='oryx', principal='oryx/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:49750
I20260620 20:09:10.095077 13287 tablet_service.cc:1514] Processing DeleteTablet for tablet 95c2b842fb614b49beefc98dfab94837 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:09:10 UTC) from {username='oryx', principal='oryx/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:33930
I20260620 20:09:10.096076 13547 tablet_replica.cc:330] T 95c2b842fb614b49beefc98dfab94837 P bceef69fdc22452ca520d306ac58656f: stopping tablet replica
I20260620 20:09:10.096446 13547 raft_consensus.cc:2229] T 95c2b842fb614b49beefc98dfab94837 P bceef69fdc22452ca520d306ac58656f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:09:10.096491 13546 tablet_replica.cc:330] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: stopping tablet replica
I20260620 20:09:10.096480 13009 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51028:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 20:09:10.096804 13547 raft_consensus.cc:2258] T 95c2b842fb614b49beefc98dfab94837 P bceef69fdc22452ca520d306ac58656f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260620 20:09:10.096922 13009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:09:10.097090 13546 raft_consensus.cc:2229] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 1 LEADER]: Raft consensus shutting down.
I20260620 20:09:10.097512 13546 raft_consensus.cc:2258] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:09:10.098730 13547 ts_tablet_manager.cc:1905] T 95c2b842fb614b49beefc98dfab94837 P bceef69fdc22452ca520d306ac58656f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:09:10.099750 13546 ts_tablet_manager.cc:1905] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:09:10.103212 13547 ts_tablet_manager.cc:1918] T 95c2b842fb614b49beefc98dfab94837 P bceef69fdc22452ca520d306ac58656f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:09:10.103346 13547 log.cc:1196] T 95c2b842fb614b49beefc98dfab94837 P bceef69fdc22452ca520d306ac58656f: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/95c2b842fb614b49beefc98dfab94837
I20260620 20:09:10.103670 13546 ts_tablet_manager.cc:1918] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:09:10.103725 13147 tablet_service.cc:1467] Processing CreateTablet for tablet 0f2913a5c76b44338f0b1bbf6d43e533 (DEFAULT_TABLE table=test-table [id=57c9f4fe48e64ae388dc27a5afc067d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:09:10.104063 13546 log.cc:1196] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-0/wal/wals/95c2b842fb614b49beefc98dfab94837
I20260620 20:09:10.104012 13287 tablet_service.cc:1467] Processing CreateTablet for tablet 0f2913a5c76b44338f0b1bbf6d43e533 (DEFAULT_TABLE table=test-table [id=57c9f4fe48e64ae388dc27a5afc067d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:09:10.104132 13147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f2913a5c76b44338f0b1bbf6d43e533. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:10.104269 13287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f2913a5c76b44338f0b1bbf6d43e533. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:10.104609 13546 ts_tablet_manager.cc:1939] T 95c2b842fb614b49beefc98dfab94837 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Deleting consensus metadata
I20260620 20:09:10.105890 12994 catalog_manager.cc:4841] TS 2402c8bc50de4c2e93d2fa5e8ebc3a2d (127.5.91.129:43491): tablet 95c2b842fb614b49beefc98dfab94837 (table test-table [id=484777c6c0a04ec3a110e09189475d87]) successfully deleted
I20260620 20:09:10.106383 13521 tablet_bootstrap.cc:492] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Bootstrap starting.
I20260620 20:09:10.106405 13547 ts_tablet_manager.cc:1939] T 95c2b842fb614b49beefc98dfab94837 P bceef69fdc22452ca520d306ac58656f: Deleting consensus metadata
I20260620 20:09:10.106760 13427 tablet_service.cc:1467] Processing CreateTablet for tablet 0f2913a5c76b44338f0b1bbf6d43e533 (DEFAULT_TABLE table=test-table [id=57c9f4fe48e64ae388dc27a5afc067d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:09:10.107024 13427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f2913a5c76b44338f0b1bbf6d43e533. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:10.107556 13521 tablet_bootstrap.cc:654] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:10.107762 12997 catalog_manager.cc:4841] TS bceef69fdc22452ca520d306ac58656f (127.5.91.131:33291): tablet 95c2b842fb614b49beefc98dfab94837 (table test-table [id=484777c6c0a04ec3a110e09189475d87]) successfully deleted
I20260620 20:09:10.108530 13545 tablet_replica.cc:330] T 95c2b842fb614b49beefc98dfab94837 P c8a411d4fd0d48df8446d518e2d71a77: stopping tablet replica
I20260620 20:09:10.108649 13523 tablet_bootstrap.cc:492] T 0f2913a5c76b44338f0b1bbf6d43e533 P c8a411d4fd0d48df8446d518e2d71a77: Bootstrap starting.
I20260620 20:09:10.108891 13545 raft_consensus.cc:2229] T 95c2b842fb614b49beefc98dfab94837 P c8a411d4fd0d48df8446d518e2d71a77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:09:10.109221 13545 raft_consensus.cc:2258] T 95c2b842fb614b49beefc98dfab94837 P c8a411d4fd0d48df8446d518e2d71a77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:09:10.109359 13521 tablet_bootstrap.cc:492] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: No bootstrap required, opened a new log
I20260620 20:09:10.109431 13521 ts_tablet_manager.cc:1397] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 20:09:10.109683 13523 tablet_bootstrap.cc:654] T 0f2913a5c76b44338f0b1bbf6d43e533 P c8a411d4fd0d48df8446d518e2d71a77: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:10.109915 13521 raft_consensus.cc:348] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.110018 13521 raft_consensus.cc:374] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:10.110046 13521 raft_consensus.cc:729] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2402c8bc50de4c2e93d2fa5e8ebc3a2d, State: Initialized, Role: FOLLOWER
I20260620 20:09:10.110214 13521 consensus_queue.cc:260] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.110846 13521 ts_tablet_manager.cc:1428] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 20:09:10.111132 13522 tablet_bootstrap.cc:492] T 0f2913a5c76b44338f0b1bbf6d43e533 P bceef69fdc22452ca520d306ac58656f: Bootstrap starting.
I20260620 20:09:10.111362 13545 ts_tablet_manager.cc:1905] T 95c2b842fb614b49beefc98dfab94837 P c8a411d4fd0d48df8446d518e2d71a77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:09:10.112064 13522 tablet_bootstrap.cc:654] T 0f2913a5c76b44338f0b1bbf6d43e533 P bceef69fdc22452ca520d306ac58656f: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:10.114573 13545 ts_tablet_manager.cc:1918] T 95c2b842fb614b49beefc98dfab94837 P c8a411d4fd0d48df8446d518e2d71a77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:09:10.114889 13522 tablet_bootstrap.cc:492] T 0f2913a5c76b44338f0b1bbf6d43e533 P bceef69fdc22452ca520d306ac58656f: No bootstrap required, opened a new log
I20260620 20:09:10.114877 13545 log.cc:1196] T 95c2b842fb614b49beefc98dfab94837 P c8a411d4fd0d48df8446d518e2d71a77: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-1/wal/wals/95c2b842fb614b49beefc98dfab94837
I20260620 20:09:10.115108 13522 ts_tablet_manager.cc:1397] T 0f2913a5c76b44338f0b1bbf6d43e533 P bceef69fdc22452ca520d306ac58656f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260620 20:09:10.115463 13545 ts_tablet_manager.cc:1939] T 95c2b842fb614b49beefc98dfab94837 P c8a411d4fd0d48df8446d518e2d71a77: Deleting consensus metadata
I20260620 20:09:10.115638 13522 raft_consensus.cc:348] T 0f2913a5c76b44338f0b1bbf6d43e533 P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.115756 13522 raft_consensus.cc:374] T 0f2913a5c76b44338f0b1bbf6d43e533 P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:10.115792 13522 raft_consensus.cc:729] T 0f2913a5c76b44338f0b1bbf6d43e533 P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bceef69fdc22452ca520d306ac58656f, State: Initialized, Role: FOLLOWER
I20260620 20:09:10.115912 13522 consensus_queue.cc:260] T 0f2913a5c76b44338f0b1bbf6d43e533 P bceef69fdc22452ca520d306ac58656f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.116168 13522 ts_tablet_manager.cc:1428] T 0f2913a5c76b44338f0b1bbf6d43e533 P bceef69fdc22452ca520d306ac58656f: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260620 20:09:10.116820 12994 catalog_manager.cc:4841] TS c8a411d4fd0d48df8446d518e2d71a77 (127.5.91.130:40845): tablet 95c2b842fb614b49beefc98dfab94837 (table test-table [id=484777c6c0a04ec3a110e09189475d87]) successfully deleted
I20260620 20:09:10.117691 13523 tablet_bootstrap.cc:492] T 0f2913a5c76b44338f0b1bbf6d43e533 P c8a411d4fd0d48df8446d518e2d71a77: No bootstrap required, opened a new log
I20260620 20:09:10.117782 13523 ts_tablet_manager.cc:1397] T 0f2913a5c76b44338f0b1bbf6d43e533 P c8a411d4fd0d48df8446d518e2d71a77: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20260620 20:09:10.118211 13523 raft_consensus.cc:348] T 0f2913a5c76b44338f0b1bbf6d43e533 P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.118309 13523 raft_consensus.cc:374] T 0f2913a5c76b44338f0b1bbf6d43e533 P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:10.118330 13523 raft_consensus.cc:729] T 0f2913a5c76b44338f0b1bbf6d43e533 P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8a411d4fd0d48df8446d518e2d71a77, State: Initialized, Role: FOLLOWER
I20260620 20:09:10.118438 13523 consensus_queue.cc:260] T 0f2913a5c76b44338f0b1bbf6d43e533 P c8a411d4fd0d48df8446d518e2d71a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.118678 13523 ts_tablet_manager.cc:1428] T 0f2913a5c76b44338f0b1bbf6d43e533 P c8a411d4fd0d48df8446d518e2d71a77: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 20:09:10.266023 13541 raft_consensus.cc:482] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:09:10.266201 13541 raft_consensus.cc:504] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.266681 13541 leader_election.cc:288] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8a411d4fd0d48df8446d518e2d71a77 (127.5.91.130:40845), bceef69fdc22452ca520d306ac58656f (127.5.91.131:33291)
I20260620 20:09:10.267372 13307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f2913a5c76b44338f0b1bbf6d43e533" candidate_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8a411d4fd0d48df8446d518e2d71a77" is_pre_election: true
I20260620 20:09:10.267372 13447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f2913a5c76b44338f0b1bbf6d43e533" candidate_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bceef69fdc22452ca520d306ac58656f" is_pre_election: true
I20260620 20:09:10.267541 13447 raft_consensus.cc:2454] T 0f2913a5c76b44338f0b1bbf6d43e533 P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2402c8bc50de4c2e93d2fa5e8ebc3a2d in term 0.
I20260620 20:09:10.267541 13307 raft_consensus.cc:2454] T 0f2913a5c76b44338f0b1bbf6d43e533 P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2402c8bc50de4c2e93d2fa5e8ebc3a2d in term 0.
I20260620 20:09:10.267943 13098 leader_election.cc:302] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2402c8bc50de4c2e93d2fa5e8ebc3a2d, bceef69fdc22452ca520d306ac58656f, c8a411d4fd0d48df8446d518e2d71a77; no voters:
I20260620 20:09:10.268141 13541 raft_consensus.cc:2781] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:09:10.268208 13541 raft_consensus.cc:482] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:09:10.268239 13541 raft_consensus.cc:3037] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:10.269194 13541 raft_consensus.cc:504] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.269542 13541 leader_election.cc:288] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [CANDIDATE]: Term 1 election: Requested vote from peers c8a411d4fd0d48df8446d518e2d71a77 (127.5.91.130:40845), bceef69fdc22452ca520d306ac58656f (127.5.91.131:33291)
I20260620 20:09:10.269897 13307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f2913a5c76b44338f0b1bbf6d43e533" candidate_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8a411d4fd0d48df8446d518e2d71a77"
I20260620 20:09:10.270036 13307 raft_consensus.cc:3037] T 0f2913a5c76b44338f0b1bbf6d43e533 P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:10.270013 13447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f2913a5c76b44338f0b1bbf6d43e533" candidate_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bceef69fdc22452ca520d306ac58656f"
I20260620 20:09:10.270124 13447 raft_consensus.cc:3037] T 0f2913a5c76b44338f0b1bbf6d43e533 P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:10.271193 13307 raft_consensus.cc:2454] T 0f2913a5c76b44338f0b1bbf6d43e533 P c8a411d4fd0d48df8446d518e2d71a77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2402c8bc50de4c2e93d2fa5e8ebc3a2d in term 1.
I20260620 20:09:10.271195 13447 raft_consensus.cc:2454] T 0f2913a5c76b44338f0b1bbf6d43e533 P bceef69fdc22452ca520d306ac58656f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2402c8bc50de4c2e93d2fa5e8ebc3a2d in term 1.
I20260620 20:09:10.271649 13098 leader_election.cc:302] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2402c8bc50de4c2e93d2fa5e8ebc3a2d, c8a411d4fd0d48df8446d518e2d71a77; no voters:
I20260620 20:09:10.271837 13541 raft_consensus.cc:2781] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:09:10.271938 13541 raft_consensus.cc:686] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 1 LEADER]: Becoming Leader. State: Replica: 2402c8bc50de4c2e93d2fa5e8ebc3a2d, State: Running, Role: LEADER
I20260620 20:09:10.272078 13541 consensus_queue.cc:237] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.273901 13009 catalog_manager.cc:5495] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2402c8bc50de4c2e93d2fa5e8ebc3a2d (127.5.91.129). New cstate: current_term: 1 leader_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } health_report { overall_health: UNKNOWN } } }
Jun 20 20:09:10 dist-test-slave-l8h4 krb5kdc[12965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986148, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.190@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM
I20260620 20:09:10.335217 13557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:10.324079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:36905 (local address 127.0.0.1:51036)
0620 20:09:10.324422 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:10.324435 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:10.324549 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:10.325192 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:10.325194 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:10.325603 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:10.325702 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:10.325709 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.326646 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.326649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:10.327221 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:10.327231 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.327954 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.327959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:10.328075 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:10.328819 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:10.328832 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:10.330613 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:10.332986 (+ 2373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:10.332998 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:10.333003 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:10.333274 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:10.333627 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:10.333630 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:10.333631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:10.333736 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:10.334588 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:10.334600 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:10.334753 (+ 153us) client_negotiation.cc:769] Sending connection context
0620 20:09:10.334865 (+ 112us) client_negotiation.cc:240] Negotiation successful
0620 20:09:10.335073 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":91,"threads_started":1}
I20260620 20:09:10.335394 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:10.324202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51036 (local address 127.5.91.190:36905)
0620 20:09:10.324318 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:10.324322 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:10.324538 (+ 216us) server_negotiation.cc:407] Connection header received
0620 20:09:10.324764 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:10.324769 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:10.324851 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:10.325175 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:10.325886 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.326466 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.327478 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.327706 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.327808 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:10.330877 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:10.330881 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:10.330882 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:10.330906 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:10.332806 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:10.333394 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:10.333397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:10.333398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:10.333477 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:10.333970 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:10.333975 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:10.333977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:10.334391 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:10.334680 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:10.335001 (+ 321us) server_negotiation.cc:299] Negotiation successful
0620 20:09:10.335206 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260620 20:09:10.336799 13494 tablet.cc:2369] T 0f2913a5c76b44338f0b1bbf6d43e533 P bceef69fdc22452ca520d306ac58656f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:09:10.340778 13009 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:51036:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 20:09:10.341521 13009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260620 20:09:10.346434 13214 tablet.cc:2369] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:09:10.347383 13147 tablet_service.cc:1467] Processing CreateTablet for tablet 568a2b8ba690423fbd686fd5eb04677d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4ac6b445041547be88a62268008717f9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 20:09:10.347469 13287 tablet_service.cc:1467] Processing CreateTablet for tablet 568a2b8ba690423fbd686fd5eb04677d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4ac6b445041547be88a62268008717f9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 20:09:10.347744 13147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 568a2b8ba690423fbd686fd5eb04677d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:10.347693 13427 tablet_service.cc:1467] Processing CreateTablet for tablet 568a2b8ba690423fbd686fd5eb04677d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4ac6b445041547be88a62268008717f9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 20:09:10.347841 13287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 568a2b8ba690423fbd686fd5eb04677d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:10.348011 13427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 568a2b8ba690423fbd686fd5eb04677d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:10.351562 13522 tablet_bootstrap.cc:492] T 568a2b8ba690423fbd686fd5eb04677d P bceef69fdc22452ca520d306ac58656f: Bootstrap starting.
I20260620 20:09:10.351562 13523 tablet_bootstrap.cc:492] T 568a2b8ba690423fbd686fd5eb04677d P c8a411d4fd0d48df8446d518e2d71a77: Bootstrap starting.
I20260620 20:09:10.352676 13522 tablet_bootstrap.cc:654] T 568a2b8ba690423fbd686fd5eb04677d P bceef69fdc22452ca520d306ac58656f: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:10.352676 13523 tablet_bootstrap.cc:654] T 568a2b8ba690423fbd686fd5eb04677d P c8a411d4fd0d48df8446d518e2d71a77: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:10.354017 13523 tablet_bootstrap.cc:492] T 568a2b8ba690423fbd686fd5eb04677d P c8a411d4fd0d48df8446d518e2d71a77: No bootstrap required, opened a new log
I20260620 20:09:10.354120 13523 ts_tablet_manager.cc:1397] T 568a2b8ba690423fbd686fd5eb04677d P c8a411d4fd0d48df8446d518e2d71a77: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 20:09:10.354828 13523 raft_consensus.cc:348] T 568a2b8ba690423fbd686fd5eb04677d P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.354955 13523 raft_consensus.cc:374] T 568a2b8ba690423fbd686fd5eb04677d P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:10.354993 13523 raft_consensus.cc:729] T 568a2b8ba690423fbd686fd5eb04677d P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8a411d4fd0d48df8446d518e2d71a77, State: Initialized, Role: FOLLOWER
I20260620 20:09:10.355726 13522 tablet_bootstrap.cc:492] T 568a2b8ba690423fbd686fd5eb04677d P bceef69fdc22452ca520d306ac58656f: No bootstrap required, opened a new log
I20260620 20:09:10.356216 13521 tablet_bootstrap.cc:492] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Bootstrap starting.
I20260620 20:09:10.356237 13522 ts_tablet_manager.cc:1397] T 568a2b8ba690423fbd686fd5eb04677d P bceef69fdc22452ca520d306ac58656f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 20:09:10.357002 13522 raft_consensus.cc:348] T 568a2b8ba690423fbd686fd5eb04677d P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.357121 13522 raft_consensus.cc:374] T 568a2b8ba690423fbd686fd5eb04677d P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:10.357154 13522 raft_consensus.cc:729] T 568a2b8ba690423fbd686fd5eb04677d P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bceef69fdc22452ca520d306ac58656f, State: Initialized, Role: FOLLOWER
I20260620 20:09:10.357277 13522 consensus_queue.cc:260] T 568a2b8ba690423fbd686fd5eb04677d P bceef69fdc22452ca520d306ac58656f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.357560 13522 ts_tablet_manager.cc:1428] T 568a2b8ba690423fbd686fd5eb04677d P bceef69fdc22452ca520d306ac58656f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 20:09:10.357707 13521 tablet_bootstrap.cc:654] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:10.357677 13527 tablet_replica.cc:439] T 568a2b8ba690423fbd686fd5eb04677d P bceef69fdc22452ca520d306ac58656f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } } }
I20260620 20:09:10.357839 13527 tablet_replica.cc:442] T 568a2b8ba690423fbd686fd5eb04677d P bceef69fdc22452ca520d306ac58656f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 20:09:10.357910 13523 consensus_queue.cc:260] T 568a2b8ba690423fbd686fd5eb04677d P c8a411d4fd0d48df8446d518e2d71a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.358235 13529 tablet_replica.cc:439] T 568a2b8ba690423fbd686fd5eb04677d P c8a411d4fd0d48df8446d518e2d71a77: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } } }
I20260620 20:09:10.358361 13529 tablet_replica.cc:442] T 568a2b8ba690423fbd686fd5eb04677d P c8a411d4fd0d48df8446d518e2d71a77: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 20:09:10.359985 13523 ts_tablet_manager.cc:1428] T 568a2b8ba690423fbd686fd5eb04677d P c8a411d4fd0d48df8446d518e2d71a77: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260620 20:09:10.360356 13521 tablet_bootstrap.cc:492] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: No bootstrap required, opened a new log
I20260620 20:09:10.360539 13521 ts_tablet_manager.cc:1397] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260620 20:09:10.361099 13521 raft_consensus.cc:348] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.361246 13521 raft_consensus.cc:374] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:10.361277 13521 raft_consensus.cc:729] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2402c8bc50de4c2e93d2fa5e8ebc3a2d, State: Initialized, Role: FOLLOWER
I20260620 20:09:10.361424 13521 consensus_queue.cc:260] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.361717 13541 tablet_replica.cc:439] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } } }
I20260620 20:09:10.361892 13541 tablet_replica.cc:442] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 20:09:10.362025 13521 ts_tablet_manager.cc:1428] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 20:09:10.604096 13541 raft_consensus.cc:482] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:09:10.604349 13541 raft_consensus.cc:504] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.605177 13541 leader_election.cc:288] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8a411d4fd0d48df8446d518e2d71a77 (127.5.91.130:40845), bceef69fdc22452ca520d306ac58656f (127.5.91.131:33291)
I20260620 20:09:10.605921 13307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "568a2b8ba690423fbd686fd5eb04677d" candidate_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8a411d4fd0d48df8446d518e2d71a77" is_pre_election: true
I20260620 20:09:10.606086 13307 raft_consensus.cc:2454] T 568a2b8ba690423fbd686fd5eb04677d P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2402c8bc50de4c2e93d2fa5e8ebc3a2d in term 0.
I20260620 20:09:10.605962 13447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "568a2b8ba690423fbd686fd5eb04677d" candidate_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bceef69fdc22452ca520d306ac58656f" is_pre_election: true
I20260620 20:09:10.606303 13447 raft_consensus.cc:2454] T 568a2b8ba690423fbd686fd5eb04677d P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2402c8bc50de4c2e93d2fa5e8ebc3a2d in term 0.
I20260620 20:09:10.606479 13098 leader_election.cc:302] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2402c8bc50de4c2e93d2fa5e8ebc3a2d, c8a411d4fd0d48df8446d518e2d71a77; no voters:
I20260620 20:09:10.606689 13541 raft_consensus.cc:2781] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:09:10.606757 13541 raft_consensus.cc:482] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:09:10.606803 13541 raft_consensus.cc:3037] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:10.607821 13541 raft_consensus.cc:504] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.608161 13541 leader_election.cc:288] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [CANDIDATE]: Term 1 election: Requested vote from peers c8a411d4fd0d48df8446d518e2d71a77 (127.5.91.130:40845), bceef69fdc22452ca520d306ac58656f (127.5.91.131:33291)
I20260620 20:09:10.608605 13307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "568a2b8ba690423fbd686fd5eb04677d" candidate_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8a411d4fd0d48df8446d518e2d71a77"
I20260620 20:09:10.608727 13307 raft_consensus.cc:3037] T 568a2b8ba690423fbd686fd5eb04677d P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:10.608810 13447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "568a2b8ba690423fbd686fd5eb04677d" candidate_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bceef69fdc22452ca520d306ac58656f"
I20260620 20:09:10.608939 13447 raft_consensus.cc:3037] T 568a2b8ba690423fbd686fd5eb04677d P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:10.609810 13307 raft_consensus.cc:2454] T 568a2b8ba690423fbd686fd5eb04677d P c8a411d4fd0d48df8446d518e2d71a77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2402c8bc50de4c2e93d2fa5e8ebc3a2d in term 1.
I20260620 20:09:10.609935 13447 raft_consensus.cc:2454] T 568a2b8ba690423fbd686fd5eb04677d P bceef69fdc22452ca520d306ac58656f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2402c8bc50de4c2e93d2fa5e8ebc3a2d in term 1.
I20260620 20:09:10.610201 13098 leader_election.cc:302] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2402c8bc50de4c2e93d2fa5e8ebc3a2d, c8a411d4fd0d48df8446d518e2d71a77; no voters:
I20260620 20:09:10.610376 13541 raft_consensus.cc:2781] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:09:10.610592 13541 raft_consensus.cc:686] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 1 LEADER]: Becoming Leader. State: Replica: 2402c8bc50de4c2e93d2fa5e8ebc3a2d, State: Running, Role: LEADER
I20260620 20:09:10.610755 13541 consensus_queue.cc:237] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.611498 13528 tablet_replica.cc:439] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: TxnStatusTablet state changed. Reason: New leader 2402c8bc50de4c2e93d2fa5e8ebc3a2d. Latest consensus state: current_term: 1 leader_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } } }
I20260620 20:09:10.611610 13528 tablet_replica.cc:442] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: This TxnStatusTablet replica's current role is: LEADER
I20260620 20:09:10.612012 13562 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 20:09:10.612174 13562 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 20:09:10.612960 13009 catalog_manager.cc:5495] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2402c8bc50de4c2e93d2fa5e8ebc3a2d (127.5.91.129). New cstate: current_term: 1 leader_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } health_report { overall_health: UNKNOWN } } }
I20260620 20:09:10.662916 13565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:10.658409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49782 (local address 127.5.91.129:43491)
0620 20:09:10.658657 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:10.658660 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:10.658673 (+ 13us) server_negotiation.cc:407] Connection header received
0620 20:09:10.658720 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:10.658722 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:10.658825 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:10.658912 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:09:10.659202 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.659757 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.660965 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.661369 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.661546 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:10.662290 (+ 744us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:09:10.662382 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:09:10.662452 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:10.662722 (+ 270us) server_negotiation.cc:299] Negotiation successful
0620 20:09:10.662784 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":85,"threads_started":1}
I20260620 20:09:10.662963 13557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:10.658108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:43491 (local address 127.0.0.1:49782)
0620 20:09:10.658400 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:10.658416 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:10.658530 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:10.658950 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:10.658953 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:10.658963 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 20:09:10.659074 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:10.659081 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.659895 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.659898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:10.660834 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:10.660845 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.661667 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.661673 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:10.661802 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:10.661894 (+ 92us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 20:09:10.662512 (+ 618us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 20:09:10.662531 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 20:09:10.662698 (+ 167us) client_negotiation.cc:240] Negotiation successful
0620 20:09:10.662816 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182}
I20260620 20:09:10.666095 13447 raft_consensus.cc:1261] T 568a2b8ba690423fbd686fd5eb04677d P bceef69fdc22452ca520d306ac58656f [term 1 FOLLOWER]: Refusing update from remote peer 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:09:10.666157 13307 raft_consensus.cc:1261] T 568a2b8ba690423fbd686fd5eb04677d P c8a411d4fd0d48df8446d518e2d71a77 [term 1 FOLLOWER]: Refusing update from remote peer 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:09:10.666548 13541 consensus_queue.cc:1035] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:09:10.666685 13528 consensus_queue.cc:1035] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:09:10.668027 13527 tablet_replica.cc:439] T 568a2b8ba690423fbd686fd5eb04677d P bceef69fdc22452ca520d306ac58656f: TxnStatusTablet state changed. Reason: New leader 2402c8bc50de4c2e93d2fa5e8ebc3a2d. Latest consensus state: current_term: 1 leader_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } } }
I20260620 20:09:10.668143 13527 tablet_replica.cc:442] T 568a2b8ba690423fbd686fd5eb04677d P bceef69fdc22452ca520d306ac58656f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 20:09:10.668854 13529 tablet_replica.cc:439] T 568a2b8ba690423fbd686fd5eb04677d P c8a411d4fd0d48df8446d518e2d71a77: TxnStatusTablet state changed. Reason: New leader 2402c8bc50de4c2e93d2fa5e8ebc3a2d. Latest consensus state: current_term: 1 leader_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } } }
I20260620 20:09:10.668960 13529 tablet_replica.cc:442] T 568a2b8ba690423fbd686fd5eb04677d P c8a411d4fd0d48df8446d518e2d71a77: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 20:09:10.670392 13528 tablet_replica.cc:439] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } } }
I20260620 20:09:10.670506 13528 tablet_replica.cc:442] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: This TxnStatusTablet replica's current role is: LEADER
I20260620 20:09:10.671026 13568 mvcc.cc:205] Tried to move back new op lower bound from 7299015273123753984 to 7299015272902914048. Current Snapshot: MvccSnapshot[applied={T|T < 7299015273123753984}]
I20260620 20:09:10.671593 13529 tablet_replica.cc:439] T 568a2b8ba690423fbd686fd5eb04677d P c8a411d4fd0d48df8446d518e2d71a77: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } } }
I20260620 20:09:10.671707 13529 tablet_replica.cc:442] T 568a2b8ba690423fbd686fd5eb04677d P c8a411d4fd0d48df8446d518e2d71a77: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 20:09:10.673602 13528 tablet_replica.cc:439] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } } }
I20260620 20:09:10.673707 13528 tablet_replica.cc:442] T 568a2b8ba690423fbd686fd5eb04677d P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: This TxnStatusTablet replica's current role is: LEADER
I20260620 20:09:10.674795 13527 tablet_replica.cc:439] T 568a2b8ba690423fbd686fd5eb04677d P bceef69fdc22452ca520d306ac58656f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } } }
I20260620 20:09:10.674925 13527 tablet_replica.cc:442] T 568a2b8ba690423fbd686fd5eb04677d P bceef69fdc22452ca520d306ac58656f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 20:09:10.688033 13565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:10.683764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:45821 (local address 127.5.91.129:43491)
0620 20:09:10.683993 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:10.684000 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:10.684447 (+ 447us) server_negotiation.cc:407] Connection header received
0620 20:09:10.684754 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:10.684758 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:10.684816 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:10.684892 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:09:10.685103 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.685883 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.686812 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.687144 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.687248 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:10.687496 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:09:10.687578 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:09:10.687645 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:10.687778 (+ 133us) server_negotiation.cc:299] Negotiation successful
0620 20:09:10.687850 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":44}
I20260620 20:09:10.688042 13577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:10.683641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:43491 (local address 127.5.91.129:45821)
0620 20:09:10.684283 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:10.684309 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:10.684505 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:10.684911 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:10.684914 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:10.684923 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 20:09:10.685018 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:10.685023 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.686018 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.686021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:10.686714 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:10.686723 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.687252 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.687254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:10.687318 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:10.687377 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 20:09:10.687687 (+ 310us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 20:09:10.687694 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 20:09:10.687761 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 20:09:10.687865 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":22,"thread_start_us":106,"threads_started":1}
I20260620 20:09:10.695562 13307 raft_consensus.cc:1261] T 0f2913a5c76b44338f0b1bbf6d43e533 P c8a411d4fd0d48df8446d518e2d71a77 [term 1 FOLLOWER]: Refusing update from remote peer 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:09:10.695642 13447 raft_consensus.cc:1261] T 0f2913a5c76b44338f0b1bbf6d43e533 P bceef69fdc22452ca520d306ac58656f [term 1 FOLLOWER]: Refusing update from remote peer 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:09:10.695950 13541 consensus_queue.cc:1035] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:09:10.696108 13528 consensus_queue.cc:1035] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:09:10.700773 13566 mvcc.cc:205] Tried to move back new op lower bound from 7299015273244733440 to 7299015271515897856. Current Snapshot: MvccSnapshot[applied={T|T < 7299015273244733440}]
I20260620 20:09:10.701062 13567 mvcc.cc:205] Tried to move back new op lower bound from 7299015273244733440 to 7299015271515897856. Current Snapshot: MvccSnapshot[applied={T|T < 7299015273244733440}]
I20260620 20:09:10.740774 13008 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:09:10.740957 13008 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:09:10.743050 13008 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170: Sending DeleteTablet for 3 replicas of tablet 0f2913a5c76b44338f0b1bbf6d43e533
I20260620 20:09:10.743870 13427 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f2913a5c76b44338f0b1bbf6d43e533 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:09:10 UTC) from {username='oryx', principal='oryx/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:53252
I20260620 20:09:10.744269 13146 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f2913a5c76b44338f0b1bbf6d43e533 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:09:10 UTC) from {username='oryx', principal='oryx/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:49750
I20260620 20:09:10.744383 13581 tablet_replica.cc:330] T 0f2913a5c76b44338f0b1bbf6d43e533 P bceef69fdc22452ca520d306ac58656f: stopping tablet replica
I20260620 20:09:10.744560 13581 raft_consensus.cc:2229] T 0f2913a5c76b44338f0b1bbf6d43e533 P bceef69fdc22452ca520d306ac58656f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:09:10.744697 13581 raft_consensus.cc:2258] T 0f2913a5c76b44338f0b1bbf6d43e533 P bceef69fdc22452ca520d306ac58656f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:09:10.744756 13582 tablet_replica.cc:330] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: stopping tablet replica
I20260620 20:09:10.744922 13582 raft_consensus.cc:2229] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 1 LEADER]: Raft consensus shutting down.
I20260620 20:09:10.745165 13582 raft_consensus.cc:2258] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:09:10.745554 13581 ts_tablet_manager.cc:1905] T 0f2913a5c76b44338f0b1bbf6d43e533 P bceef69fdc22452ca520d306ac58656f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:09:10.745843 13582 ts_tablet_manager.cc:1905] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:09:10.745865 13287 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f2913a5c76b44338f0b1bbf6d43e533 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:09:10 UTC) from {username='oryx', principal='oryx/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:33930
I20260620 20:09:10.748147 13584 tablet_replica.cc:330] T 0f2913a5c76b44338f0b1bbf6d43e533 P c8a411d4fd0d48df8446d518e2d71a77: stopping tablet replica
I20260620 20:09:10.748349 13584 raft_consensus.cc:2229] T 0f2913a5c76b44338f0b1bbf6d43e533 P c8a411d4fd0d48df8446d518e2d71a77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:09:10.748567 13584 raft_consensus.cc:2258] T 0f2913a5c76b44338f0b1bbf6d43e533 P c8a411d4fd0d48df8446d518e2d71a77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:09:10.749038 13581 ts_tablet_manager.cc:1918] T 0f2913a5c76b44338f0b1bbf6d43e533 P bceef69fdc22452ca520d306ac58656f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 20:09:10.749075 13582 ts_tablet_manager.cc:1918] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 20:09:10.749137 13582 log.cc:1196] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-0/wal/wals/0f2913a5c76b44338f0b1bbf6d43e533
I20260620 20:09:10.749244 13581 log.cc:1196] T 0f2913a5c76b44338f0b1bbf6d43e533 P bceef69fdc22452ca520d306ac58656f: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/0f2913a5c76b44338f0b1bbf6d43e533
I20260620 20:09:10.749346 13584 ts_tablet_manager.cc:1905] T 0f2913a5c76b44338f0b1bbf6d43e533 P c8a411d4fd0d48df8446d518e2d71a77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:09:10.749569 13581 ts_tablet_manager.cc:1939] T 0f2913a5c76b44338f0b1bbf6d43e533 P bceef69fdc22452ca520d306ac58656f: Deleting consensus metadata
I20260620 20:09:10.750566 12997 catalog_manager.cc:4841] TS bceef69fdc22452ca520d306ac58656f (127.5.91.131:33291): tablet 0f2913a5c76b44338f0b1bbf6d43e533 (table test-table [id=57c9f4fe48e64ae388dc27a5afc067d7]) successfully deleted
I20260620 20:09:10.750741 13582 ts_tablet_manager.cc:1939] T 0f2913a5c76b44338f0b1bbf6d43e533 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Deleting consensus metadata
I20260620 20:09:10.751461 12994 catalog_manager.cc:4841] TS 2402c8bc50de4c2e93d2fa5e8ebc3a2d (127.5.91.129:43491): tablet 0f2913a5c76b44338f0b1bbf6d43e533 (table test-table [id=57c9f4fe48e64ae388dc27a5afc067d7]) successfully deleted
I20260620 20:09:10.752594 13584 ts_tablet_manager.cc:1918] T 0f2913a5c76b44338f0b1bbf6d43e533 P c8a411d4fd0d48df8446d518e2d71a77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 20:09:10.752681 13584 log.cc:1196] T 0f2913a5c76b44338f0b1bbf6d43e533 P c8a411d4fd0d48df8446d518e2d71a77: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-1/wal/wals/0f2913a5c76b44338f0b1bbf6d43e533
I20260620 20:09:10.752970 13584 ts_tablet_manager.cc:1939] T 0f2913a5c76b44338f0b1bbf6d43e533 P c8a411d4fd0d48df8446d518e2d71a77: Deleting consensus metadata
I20260620 20:09:10.753698 12994 catalog_manager.cc:4841] TS c8a411d4fd0d48df8446d518e2d71a77 (127.5.91.130:40845): tablet 0f2913a5c76b44338f0b1bbf6d43e533 (table test-table [id=57c9f4fe48e64ae388dc27a5afc067d7]) successfully deleted
Jun 20 20:09:10 dist-test-slave-l8h4 krb5kdc[12965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 20:09:10 dist-test-slave-l8h4 krb5kdc[12965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:10 dist-test-slave-l8h4 krb5kdc[12965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
W20260620 20:09:10.781612 13059 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:10.770744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51040 (local address 127.5.91.190:36905)
0620 20:09:10.770890 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:10.770895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:10.771048 (+ 153us) server_negotiation.cc:407] Connection header received
0620 20:09:10.771168 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:10.771171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:10.771225 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:10.771297 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:10.772299 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.772836 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.774226 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.774457 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.774541 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:10.781486 (+ 6945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51040: BlockingRecv error: recv got EOF from 127.0.0.1:51040 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 20 20:09:10 dist-test-slave-l8h4 krb5kdc[12965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM
I20260620 20:09:10.792394 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:10.784008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51044 (local address 127.5.91.190:36905)
0620 20:09:10.784145 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:10.784149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:10.784166 (+ 17us) server_negotiation.cc:407] Connection header received
0620 20:09:10.784311 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:10.784315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:10.784366 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:10.784491 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:10.785081 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.785586 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.786390 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.786614 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.786699 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:10.789394 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:10.789398 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:10.789400 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:10.789425 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:10.790884 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:10.791317 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:10.791320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:10.791322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:10.791390 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:10.791630 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:10.791633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:10.791635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:10.791806 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:10.791919 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:10.792135 (+ 216us) server_negotiation.cc:299] Negotiation successful
0620 20:09:10.792239 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260620 20:09:10.795285 13008 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51044:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:09:10.795696 13008 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:09:10.800758 13287 tablet_service.cc:1467] Processing CreateTablet for tablet 348086c8b5804212b19bd8426589c5f1 (DEFAULT_TABLE table=test-table [id=ce97c26bd261407ab4ebf5b9a7d840c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:09:10.800758 13427 tablet_service.cc:1467] Processing CreateTablet for tablet 348086c8b5804212b19bd8426589c5f1 (DEFAULT_TABLE table=test-table [id=ce97c26bd261407ab4ebf5b9a7d840c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:09:10.800758 13146 tablet_service.cc:1467] Processing CreateTablet for tablet 348086c8b5804212b19bd8426589c5f1 (DEFAULT_TABLE table=test-table [id=ce97c26bd261407ab4ebf5b9a7d840c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:09:10.801049 13427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 348086c8b5804212b19bd8426589c5f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:10.801050 13146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 348086c8b5804212b19bd8426589c5f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:10.801061 13287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 348086c8b5804212b19bd8426589c5f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:10.803231 13523 tablet_bootstrap.cc:492] T 348086c8b5804212b19bd8426589c5f1 P c8a411d4fd0d48df8446d518e2d71a77: Bootstrap starting.
I20260620 20:09:10.803781 13522 tablet_bootstrap.cc:492] T 348086c8b5804212b19bd8426589c5f1 P bceef69fdc22452ca520d306ac58656f: Bootstrap starting.
I20260620 20:09:10.803959 13521 tablet_bootstrap.cc:492] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Bootstrap starting.
I20260620 20:09:10.804257 13523 tablet_bootstrap.cc:654] T 348086c8b5804212b19bd8426589c5f1 P c8a411d4fd0d48df8446d518e2d71a77: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:10.804693 13522 tablet_bootstrap.cc:654] T 348086c8b5804212b19bd8426589c5f1 P bceef69fdc22452ca520d306ac58656f: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:10.804694 13521 tablet_bootstrap.cc:654] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:10.806176 13523 tablet_bootstrap.cc:492] T 348086c8b5804212b19bd8426589c5f1 P c8a411d4fd0d48df8446d518e2d71a77: No bootstrap required, opened a new log
I20260620 20:09:10.806267 13523 ts_tablet_manager.cc:1397] T 348086c8b5804212b19bd8426589c5f1 P c8a411d4fd0d48df8446d518e2d71a77: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260620 20:09:10.806330 13522 tablet_bootstrap.cc:492] T 348086c8b5804212b19bd8426589c5f1 P bceef69fdc22452ca520d306ac58656f: No bootstrap required, opened a new log
I20260620 20:09:10.806401 13522 ts_tablet_manager.cc:1397] T 348086c8b5804212b19bd8426589c5f1 P bceef69fdc22452ca520d306ac58656f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 20:09:10.806712 13523 raft_consensus.cc:348] T 348086c8b5804212b19bd8426589c5f1 P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.806777 13522 raft_consensus.cc:348] T 348086c8b5804212b19bd8426589c5f1 P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.806847 13523 raft_consensus.cc:374] T 348086c8b5804212b19bd8426589c5f1 P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:10.806865 13522 raft_consensus.cc:374] T 348086c8b5804212b19bd8426589c5f1 P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:10.806890 13523 raft_consensus.cc:729] T 348086c8b5804212b19bd8426589c5f1 P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8a411d4fd0d48df8446d518e2d71a77, State: Initialized, Role: FOLLOWER
I20260620 20:09:10.806895 13522 raft_consensus.cc:729] T 348086c8b5804212b19bd8426589c5f1 P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bceef69fdc22452ca520d306ac58656f, State: Initialized, Role: FOLLOWER
I20260620 20:09:10.807008 13523 consensus_queue.cc:260] T 348086c8b5804212b19bd8426589c5f1 P c8a411d4fd0d48df8446d518e2d71a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.807008 13522 consensus_queue.cc:260] T 348086c8b5804212b19bd8426589c5f1 P bceef69fdc22452ca520d306ac58656f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.807281 13523 ts_tablet_manager.cc:1428] T 348086c8b5804212b19bd8426589c5f1 P c8a411d4fd0d48df8446d518e2d71a77: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 20:09:10.807291 13522 ts_tablet_manager.cc:1428] T 348086c8b5804212b19bd8426589c5f1 P bceef69fdc22452ca520d306ac58656f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 20:09:10.807873 13521 tablet_bootstrap.cc:492] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: No bootstrap required, opened a new log
I20260620 20:09:10.807969 13521 ts_tablet_manager.cc:1397] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260620 20:09:10.808357 13521 raft_consensus.cc:348] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.808461 13521 raft_consensus.cc:374] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:10.808490 13521 raft_consensus.cc:729] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2402c8bc50de4c2e93d2fa5e8ebc3a2d, State: Initialized, Role: FOLLOWER
I20260620 20:09:10.808616 13521 consensus_queue.cc:260] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.808866 13521 ts_tablet_manager.cc:1428] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 20:09:10.877377 13573 raft_consensus.cc:482] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:09:10.877533 13573 raft_consensus.cc:504] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.878011 13573 leader_election.cc:288] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8a411d4fd0d48df8446d518e2d71a77 (127.5.91.130:40845), bceef69fdc22452ca520d306ac58656f (127.5.91.131:33291)
I20260620 20:09:10.878499 13307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "348086c8b5804212b19bd8426589c5f1" candidate_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8a411d4fd0d48df8446d518e2d71a77" is_pre_election: true
I20260620 20:09:10.878538 13447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "348086c8b5804212b19bd8426589c5f1" candidate_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bceef69fdc22452ca520d306ac58656f" is_pre_election: true
I20260620 20:09:10.878638 13307 raft_consensus.cc:2454] T 348086c8b5804212b19bd8426589c5f1 P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2402c8bc50de4c2e93d2fa5e8ebc3a2d in term 0.
I20260620 20:09:10.878654 13447 raft_consensus.cc:2454] T 348086c8b5804212b19bd8426589c5f1 P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2402c8bc50de4c2e93d2fa5e8ebc3a2d in term 0.
I20260620 20:09:10.878942 13101 leader_election.cc:302] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2402c8bc50de4c2e93d2fa5e8ebc3a2d, bceef69fdc22452ca520d306ac58656f; no voters:
I20260620 20:09:10.879092 13573 raft_consensus.cc:2781] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:09:10.879148 13573 raft_consensus.cc:482] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:09:10.879189 13573 raft_consensus.cc:3037] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:10.880182 13573 raft_consensus.cc:504] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.880486 13573 leader_election.cc:288] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [CANDIDATE]: Term 1 election: Requested vote from peers c8a411d4fd0d48df8446d518e2d71a77 (127.5.91.130:40845), bceef69fdc22452ca520d306ac58656f (127.5.91.131:33291)
I20260620 20:09:10.880797 13307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "348086c8b5804212b19bd8426589c5f1" candidate_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8a411d4fd0d48df8446d518e2d71a77"
I20260620 20:09:10.880906 13307 raft_consensus.cc:3037] T 348086c8b5804212b19bd8426589c5f1 P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:10.881278 13447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "348086c8b5804212b19bd8426589c5f1" candidate_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bceef69fdc22452ca520d306ac58656f"
I20260620 20:09:10.881400 13447 raft_consensus.cc:3037] T 348086c8b5804212b19bd8426589c5f1 P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:10.882033 13307 raft_consensus.cc:2454] T 348086c8b5804212b19bd8426589c5f1 P c8a411d4fd0d48df8446d518e2d71a77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2402c8bc50de4c2e93d2fa5e8ebc3a2d in term 1.
I20260620 20:09:10.882665 13447 raft_consensus.cc:2454] T 348086c8b5804212b19bd8426589c5f1 P bceef69fdc22452ca520d306ac58656f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2402c8bc50de4c2e93d2fa5e8ebc3a2d in term 1.
I20260620 20:09:10.882987 13098 leader_election.cc:302] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2402c8bc50de4c2e93d2fa5e8ebc3a2d, c8a411d4fd0d48df8446d518e2d71a77; no voters:
I20260620 20:09:10.883147 13573 raft_consensus.cc:2781] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:09:10.883234 13573 raft_consensus.cc:686] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 1 LEADER]: Becoming Leader. State: Replica: 2402c8bc50de4c2e93d2fa5e8ebc3a2d, State: Running, Role: LEADER
I20260620 20:09:10.883368 13573 consensus_queue.cc:237] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.885317 13009 catalog_manager.cc:5495] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2402c8bc50de4c2e93d2fa5e8ebc3a2d (127.5.91.129). New cstate: current_term: 1 leader_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } health_report { overall_health: UNKNOWN } } }
I20260620 20:09:10.911585 13565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:10.907895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49794 (local address 127.5.91.129:43491)
0620 20:09:10.908020 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:10.908024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:10.908037 (+ 13us) server_negotiation.cc:407] Connection header received
0620 20:09:10.908132 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:10.908134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:10.908178 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:10.908237 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:09:10.908534 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.909059 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.910317 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:10.910582 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:10.910681 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:10.911044 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:09:10.911121 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:09:10.911189 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:10.911373 (+ 184us) server_negotiation.cc:299] Negotiation successful
0620 20:09:10.911426 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 20:09:10.914085 13447 raft_consensus.cc:1261] T 348086c8b5804212b19bd8426589c5f1 P bceef69fdc22452ca520d306ac58656f [term 1 FOLLOWER]: Refusing update from remote peer 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:09:10.914124 13307 raft_consensus.cc:1261] T 348086c8b5804212b19bd8426589c5f1 P c8a411d4fd0d48df8446d518e2d71a77 [term 1 FOLLOWER]: Refusing update from remote peer 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:09:10.914577 13573 consensus_queue.cc:1035] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:09:10.914760 13528 consensus_queue.cc:1035] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:09:10.922964 13008 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:09:10.923115 13008 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:09:10.925154 13008 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170: Sending DeleteTablet for 3 replicas of tablet 348086c8b5804212b19bd8426589c5f1
I20260620 20:09:10.925853 13287 tablet_service.cc:1514] Processing DeleteTablet for tablet 348086c8b5804212b19bd8426589c5f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:09:10 UTC) from {username='oryx', principal='oryx/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:33930
I20260620 20:09:10.925910 13146 tablet_service.cc:1514] Processing DeleteTablet for tablet 348086c8b5804212b19bd8426589c5f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:09:10 UTC) from {username='oryx', principal='oryx/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:49750
I20260620 20:09:10.926052 13584 tablet_replica.cc:330] T 348086c8b5804212b19bd8426589c5f1 P c8a411d4fd0d48df8446d518e2d71a77: stopping tablet replica
I20260620 20:09:10.926061 13582 tablet_replica.cc:330] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: stopping tablet replica
I20260620 20:09:10.926191 13584 raft_consensus.cc:2229] T 348086c8b5804212b19bd8426589c5f1 P c8a411d4fd0d48df8446d518e2d71a77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:09:10.926199 13582 raft_consensus.cc:2229] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 1 LEADER]: Raft consensus shutting down.
I20260620 20:09:10.926260 13427 tablet_service.cc:1514] Processing DeleteTablet for tablet 348086c8b5804212b19bd8426589c5f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:09:10 UTC) from {username='oryx', principal='oryx/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:53252
I20260620 20:09:10.926405 13584 raft_consensus.cc:2258] T 348086c8b5804212b19bd8426589c5f1 P c8a411d4fd0d48df8446d518e2d71a77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:09:10.926432 13582 raft_consensus.cc:2258] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:09:10.926483 13581 tablet_replica.cc:330] T 348086c8b5804212b19bd8426589c5f1 P bceef69fdc22452ca520d306ac58656f: stopping tablet replica
I20260620 20:09:10.926585 13581 raft_consensus.cc:2229] T 348086c8b5804212b19bd8426589c5f1 P bceef69fdc22452ca520d306ac58656f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:09:10.926715 13581 raft_consensus.cc:2258] T 348086c8b5804212b19bd8426589c5f1 P bceef69fdc22452ca520d306ac58656f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:09:10.927174 13584 ts_tablet_manager.cc:1905] T 348086c8b5804212b19bd8426589c5f1 P c8a411d4fd0d48df8446d518e2d71a77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:09:10.927343 13581 ts_tablet_manager.cc:1905] T 348086c8b5804212b19bd8426589c5f1 P bceef69fdc22452ca520d306ac58656f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:09:10.927311 13008 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51044:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 20:09:10.927446 13582 ts_tablet_manager.cc:1905] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 20:09:10.927711 13008 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:09:10.929738 13581 ts_tablet_manager.cc:1918] T 348086c8b5804212b19bd8426589c5f1 P bceef69fdc22452ca520d306ac58656f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:09:10.929821 13581 log.cc:1196] T 348086c8b5804212b19bd8426589c5f1 P bceef69fdc22452ca520d306ac58656f: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/348086c8b5804212b19bd8426589c5f1
I20260620 20:09:10.930068 13581 ts_tablet_manager.cc:1939] T 348086c8b5804212b19bd8426589c5f1 P bceef69fdc22452ca520d306ac58656f: Deleting consensus metadata
I20260620 20:09:10.930320 13582 ts_tablet_manager.cc:1918] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:09:10.930387 13582 log.cc:1196] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-0/wal/wals/348086c8b5804212b19bd8426589c5f1
I20260620 20:09:10.930644 13582 ts_tablet_manager.cc:1939] T 348086c8b5804212b19bd8426589c5f1 P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Deleting consensus metadata
I20260620 20:09:10.930756 13584 ts_tablet_manager.cc:1918] T 348086c8b5804212b19bd8426589c5f1 P c8a411d4fd0d48df8446d518e2d71a77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:09:10.930835 13584 log.cc:1196] T 348086c8b5804212b19bd8426589c5f1 P c8a411d4fd0d48df8446d518e2d71a77: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-1/wal/wals/348086c8b5804212b19bd8426589c5f1
I20260620 20:09:10.930816 12997 catalog_manager.cc:4841] TS bceef69fdc22452ca520d306ac58656f (127.5.91.131:33291): tablet 348086c8b5804212b19bd8426589c5f1 (table test-table [id=ce97c26bd261407ab4ebf5b9a7d840c1]) successfully deleted
I20260620 20:09:10.931066 13584 ts_tablet_manager.cc:1939] T 348086c8b5804212b19bd8426589c5f1 P c8a411d4fd0d48df8446d518e2d71a77: Deleting consensus metadata
I20260620 20:09:10.931344 12994 catalog_manager.cc:4841] TS 2402c8bc50de4c2e93d2fa5e8ebc3a2d (127.5.91.129:43491): tablet 348086c8b5804212b19bd8426589c5f1 (table test-table [id=ce97c26bd261407ab4ebf5b9a7d840c1]) successfully deleted
I20260620 20:09:10.931870 12994 catalog_manager.cc:4841] TS c8a411d4fd0d48df8446d518e2d71a77 (127.5.91.130:40845): tablet 348086c8b5804212b19bd8426589c5f1 (table test-table [id=ce97c26bd261407ab4ebf5b9a7d840c1]) successfully deleted
I20260620 20:09:10.933894 13146 tablet_service.cc:1467] Processing CreateTablet for tablet c1a791d491d64a308d0cf6466fffd5da (DEFAULT_TABLE table=test-table [id=5bb97a9e43e9494a9b3ef91572499746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:09:10.933966 13287 tablet_service.cc:1467] Processing CreateTablet for tablet c1a791d491d64a308d0cf6466fffd5da (DEFAULT_TABLE table=test-table [id=5bb97a9e43e9494a9b3ef91572499746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:09:10.934089 13427 tablet_service.cc:1467] Processing CreateTablet for tablet c1a791d491d64a308d0cf6466fffd5da (DEFAULT_TABLE table=test-table [id=5bb97a9e43e9494a9b3ef91572499746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:09:10.934192 13287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1a791d491d64a308d0cf6466fffd5da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:10.934192 13146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1a791d491d64a308d0cf6466fffd5da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:10.934670 13427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1a791d491d64a308d0cf6466fffd5da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:10.936486 13523 tablet_bootstrap.cc:492] T c1a791d491d64a308d0cf6466fffd5da P c8a411d4fd0d48df8446d518e2d71a77: Bootstrap starting.
I20260620 20:09:10.936929 13522 tablet_bootstrap.cc:492] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f: Bootstrap starting.
I20260620 20:09:10.936866 13521 tablet_bootstrap.cc:492] T c1a791d491d64a308d0cf6466fffd5da P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Bootstrap starting.
I20260620 20:09:10.937693 13523 tablet_bootstrap.cc:654] T c1a791d491d64a308d0cf6466fffd5da P c8a411d4fd0d48df8446d518e2d71a77: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:10.937726 13521 tablet_bootstrap.cc:654] T c1a791d491d64a308d0cf6466fffd5da P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:10.937752 13522 tablet_bootstrap.cc:654] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:10.939098 13523 tablet_bootstrap.cc:492] T c1a791d491d64a308d0cf6466fffd5da P c8a411d4fd0d48df8446d518e2d71a77: No bootstrap required, opened a new log
I20260620 20:09:10.939170 13522 tablet_bootstrap.cc:492] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f: No bootstrap required, opened a new log
I20260620 20:09:10.939170 13521 tablet_bootstrap.cc:492] T c1a791d491d64a308d0cf6466fffd5da P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: No bootstrap required, opened a new log
I20260620 20:09:10.939182 13523 ts_tablet_manager.cc:1397] T c1a791d491d64a308d0cf6466fffd5da P c8a411d4fd0d48df8446d518e2d71a77: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260620 20:09:10.939258 13522 ts_tablet_manager.cc:1397] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 20:09:10.939258 13521 ts_tablet_manager.cc:1397] T c1a791d491d64a308d0cf6466fffd5da P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 20:09:10.939599 13523 raft_consensus.cc:348] T c1a791d491d64a308d0cf6466fffd5da P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.939702 13523 raft_consensus.cc:374] T c1a791d491d64a308d0cf6466fffd5da P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:10.939731 13523 raft_consensus.cc:729] T c1a791d491d64a308d0cf6466fffd5da P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8a411d4fd0d48df8446d518e2d71a77, State: Initialized, Role: FOLLOWER
I20260620 20:09:10.939688 13521 raft_consensus.cc:348] T c1a791d491d64a308d0cf6466fffd5da P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.939857 13523 consensus_queue.cc:260] T c1a791d491d64a308d0cf6466fffd5da P c8a411d4fd0d48df8446d518e2d71a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.939882 13521 raft_consensus.cc:374] T c1a791d491d64a308d0cf6466fffd5da P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:10.939936 13521 raft_consensus.cc:729] T c1a791d491d64a308d0cf6466fffd5da P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2402c8bc50de4c2e93d2fa5e8ebc3a2d, State: Initialized, Role: FOLLOWER
I20260620 20:09:10.939688 13522 raft_consensus.cc:348] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.939987 13522 raft_consensus.cc:374] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:10.940023 13522 raft_consensus.cc:729] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bceef69fdc22452ca520d306ac58656f, State: Initialized, Role: FOLLOWER
I20260620 20:09:10.940060 13521 consensus_queue.cc:260] T c1a791d491d64a308d0cf6466fffd5da P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.940107 13523 ts_tablet_manager.cc:1428] T c1a791d491d64a308d0cf6466fffd5da P c8a411d4fd0d48df8446d518e2d71a77: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260620 20:09:10.940129 13522 consensus_queue.cc:260] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:10.940323 13521 ts_tablet_manager.cc:1428] T c1a791d491d64a308d0cf6466fffd5da P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 20:09:10.940409 13522 ts_tablet_manager.cc:1428] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260620 20:09:11.288911 13527 raft_consensus.cc:482] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:09:11.289072 13527 raft_consensus.cc:504] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:11.290237 13527 leader_election.cc:288] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2402c8bc50de4c2e93d2fa5e8ebc3a2d (127.5.91.129:43491), c8a411d4fd0d48df8446d518e2d71a77 (127.5.91.130:40845)
I20260620 20:09:11.294486 13606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:11.290473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:43491 (local address 127.5.91.131:47759)
0620 20:09:11.290762 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:11.290777 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:11.290904 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:11.291168 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:11.291171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:11.291208 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:11.291333 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:11.291340 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:11.292261 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:11.292264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:11.293240 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:11.293250 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:11.293991 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:11.293995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:11.294103 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:11.294131 (+ 28us) client_negotiation.cc:769] Sending connection context
0620 20:09:11.294232 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 20:09:11.294309 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"spinlock_wait_cycles":3840,"thread_start_us":94,"threads_started":1}
I20260620 20:09:11.294745 13565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:11.290584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:47759 (local address 127.5.91.129:43491)
0620 20:09:11.290746 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:11.290751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:11.290858 (+ 107us) server_negotiation.cc:407] Connection header received
0620 20:09:11.291018 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:11.291021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:11.291070 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:11.291146 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:11.291515 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:11.292132 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:11.293360 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:11.293857 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:11.293951 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:11.294011 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:11.294332 (+ 321us) server_negotiation.cc:299] Negotiation successful
0620 20:09:11.294624 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260620 20:09:11.295012 13607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:11.290812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:40845 (local address 127.5.91.131:57979)
0620 20:09:11.291110 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:11.291122 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:11.291202 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:11.291515 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:11.291518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:11.291536 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:11.291647 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:11.291653 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:11.292654 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:11.292658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:11.293526 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:11.293536 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:11.294374 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:11.294377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:11.294659 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:11.294684 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 20:09:11.294762 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 20:09:11.294831 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"spinlock_wait_cycles":6016,"thread_start_us":93,"threads_started":1}
I20260620 20:09:11.295218 13608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:11.290948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:57979 (local address 127.5.91.130:40845)
0620 20:09:11.291146 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:11.291150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:11.291164 (+ 14us) server_negotiation.cc:407] Connection header received
0620 20:09:11.291301 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:11.291304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:11.291347 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:11.291426 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:11.291798 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:11.292490 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:11.293676 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:11.294239 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:11.294360 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:11.294421 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:11.294988 (+ 567us) server_negotiation.cc:299] Negotiation successful
0620 20:09:11.295079 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":55,"threads_started":1}
I20260620 20:09:11.295516 13166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1a791d491d64a308d0cf6466fffd5da" candidate_uuid: "bceef69fdc22452ca520d306ac58656f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" is_pre_election: true
I20260620 20:09:11.296099 13166 raft_consensus.cc:2454] T c1a791d491d64a308d0cf6466fffd5da P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bceef69fdc22452ca520d306ac58656f in term 0.
I20260620 20:09:11.296334 13307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1a791d491d64a308d0cf6466fffd5da" candidate_uuid: "bceef69fdc22452ca520d306ac58656f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8a411d4fd0d48df8446d518e2d71a77" is_pre_election: true
I20260620 20:09:11.296478 13307 raft_consensus.cc:2454] T c1a791d491d64a308d0cf6466fffd5da P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bceef69fdc22452ca520d306ac58656f in term 0.
I20260620 20:09:11.296567 13378 leader_election.cc:302] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2402c8bc50de4c2e93d2fa5e8ebc3a2d, bceef69fdc22452ca520d306ac58656f; no voters:
I20260620 20:09:11.296962 13527 raft_consensus.cc:2781] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:09:11.297025 13527 raft_consensus.cc:482] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:09:11.297053 13527 raft_consensus.cc:3037] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:11.298003 13527 raft_consensus.cc:504] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:11.298367 13527 leader_election.cc:288] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f [CANDIDATE]: Term 1 election: Requested vote from peers 2402c8bc50de4c2e93d2fa5e8ebc3a2d (127.5.91.129:43491), c8a411d4fd0d48df8446d518e2d71a77 (127.5.91.130:40845)
I20260620 20:09:11.298993 13307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1a791d491d64a308d0cf6466fffd5da" candidate_uuid: "bceef69fdc22452ca520d306ac58656f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8a411d4fd0d48df8446d518e2d71a77"
I20260620 20:09:11.299221 13307 raft_consensus.cc:3037] T c1a791d491d64a308d0cf6466fffd5da P c8a411d4fd0d48df8446d518e2d71a77 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:11.300631 13307 raft_consensus.cc:2454] T c1a791d491d64a308d0cf6466fffd5da P c8a411d4fd0d48df8446d518e2d71a77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bceef69fdc22452ca520d306ac58656f in term 1.
I20260620 20:09:11.301112 13378 leader_election.cc:302] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bceef69fdc22452ca520d306ac58656f, c8a411d4fd0d48df8446d518e2d71a77; no voters:
I20260620 20:09:11.301067 13166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1a791d491d64a308d0cf6466fffd5da" candidate_uuid: "bceef69fdc22452ca520d306ac58656f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d"
I20260620 20:09:11.301263 13166 raft_consensus.cc:3037] T c1a791d491d64a308d0cf6466fffd5da P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:11.301415 13527 raft_consensus.cc:2781] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:09:11.301769 13527 raft_consensus.cc:686] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f [term 1 LEADER]: Becoming Leader. State: Replica: bceef69fdc22452ca520d306ac58656f, State: Running, Role: LEADER
I20260620 20:09:11.302031 13527 consensus_queue.cc:237] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } }
I20260620 20:09:11.302630 13166 raft_consensus.cc:2454] T c1a791d491d64a308d0cf6466fffd5da P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bceef69fdc22452ca520d306ac58656f in term 1.
I20260620 20:09:11.304656 13010 catalog_manager.cc:5495] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f reported cstate change: term changed from 0 to 1, leader changed from <none> to bceef69fdc22452ca520d306ac58656f (127.5.91.131). New cstate: current_term: 1 leader_uuid: "bceef69fdc22452ca520d306ac58656f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bceef69fdc22452ca520d306ac58656f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33291 } health_report { overall_health: HEALTHY } } }
I20260620 20:09:11.322904 13611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:11.319133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53268 (local address 127.5.91.131:33291)
0620 20:09:11.319470 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:11.319477 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:11.319487 (+ 10us) server_negotiation.cc:407] Connection header received
0620 20:09:11.319553 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:11.319555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:11.319603 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:11.319709 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:09:11.319945 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:11.320508 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:11.321315 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:11.321573 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:11.321658 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:11.322006 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:09:11.322497 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:09:11.322614 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:11.322683 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 20:09:11.322740 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":167,"threads_started":1}
I20260620 20:09:11.336273 13565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:11.332877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:49189 (local address 127.5.91.129:43491)
0620 20:09:11.333042 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:11.333047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:11.333149 (+ 102us) server_negotiation.cc:407] Connection header received
0620 20:09:11.333306 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:11.333309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:11.333390 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:11.333453 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:09:11.333711 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:11.334289 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:11.335196 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:11.335442 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:11.335552 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:11.335845 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:09:11.335922 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:09:11.335978 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:11.336070 (+ 92us) server_negotiation.cc:299] Negotiation successful
0620 20:09:11.336122 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260620 20:09:11.336452 13614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:11.332681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:43491 (local address 127.5.91.131:49189)
0620 20:09:11.333062 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:11.333076 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:11.333206 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:11.333487 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:11.333489 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:11.333497 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 20:09:11.333608 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:11.333613 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:11.334394 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:11.334398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:11.335078 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:11.335088 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:11.335587 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:11.335591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:11.335670 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:11.335716 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 20:09:11.335996 (+ 280us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 20:09:11.336003 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 20:09:11.336061 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 20:09:11.336146 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":144,"threads_started":1}
I20260620 20:09:11.346438 13307 raft_consensus.cc:1261] T c1a791d491d64a308d0cf6466fffd5da P c8a411d4fd0d48df8446d518e2d71a77 [term 1 FOLLOWER]: Refusing update from remote peer bceef69fdc22452ca520d306ac58656f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:09:11.346563 13166 raft_consensus.cc:1261] T c1a791d491d64a308d0cf6466fffd5da P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 1 FOLLOWER]: Refusing update from remote peer bceef69fdc22452ca520d306ac58656f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:09:11.347044 13527 consensus_queue.cc:1035] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8a411d4fd0d48df8446d518e2d71a77" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:09:11.347267 13609 consensus_queue.cc:1035] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2402c8bc50de4c2e93d2fa5e8ebc3a2d" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 43491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:09:11.367383 13611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:11.363828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:33207 (local address 127.5.91.131:33291)
0620 20:09:11.363976 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:11.363980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:11.364235 (+ 255us) server_negotiation.cc:407] Connection header received
0620 20:09:11.364317 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:11.364325 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:11.364372 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:11.364434 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:09:11.364688 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:11.365404 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:11.366457 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:11.366686 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:11.366769 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:11.366987 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:09:11.367057 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:09:11.367113 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:11.367200 (+ 87us) server_negotiation.cc:299] Negotiation successful
0620 20:09:11.367247 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 20:09:11.367393 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:11.363747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:33291 (local address 127.5.91.129:33207)
0620 20:09:11.364130 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:11.364143 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:11.364236 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:11.364454 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:11.364457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:11.364464 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 20:09:11.364567 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:11.364573 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:11.365512 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:11.365516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:11.366355 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:11.366364 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:11.366779 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:11.366783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:11.366863 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:11.366890 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 20:09:11.367127 (+ 237us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 20:09:11.367134 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 20:09:11.367190 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 20:09:11.367271 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":72,"threads_started":1}
I20260620 20:09:11.390518 13009 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:09:11.390676 13009 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:09:11.392930 13009 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 89e55a1f00e44812af32fcc47b938170: Sending DeleteTablet for 3 replicas of tablet c1a791d491d64a308d0cf6466fffd5da
I20260620 20:09:11.393540 13146 tablet_service.cc:1514] Processing DeleteTablet for tablet c1a791d491d64a308d0cf6466fffd5da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:09:11 UTC) from {username='oryx', principal='oryx/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:49750
I20260620 20:09:11.393568 13427 tablet_service.cc:1514] Processing DeleteTablet for tablet c1a791d491d64a308d0cf6466fffd5da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:09:11 UTC) from {username='oryx', principal='oryx/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:53252
I20260620 20:09:11.393726 13581 tablet_replica.cc:330] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f: stopping tablet replica
I20260620 20:09:11.393736 13582 tablet_replica.cc:330] T c1a791d491d64a308d0cf6466fffd5da P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: stopping tablet replica
I20260620 20:09:11.393836 13581 raft_consensus.cc:2229] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f [term 1 LEADER]: Raft consensus shutting down.
I20260620 20:09:11.393836 13582 raft_consensus.cc:2229] T c1a791d491d64a308d0cf6466fffd5da P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:09:11.393940 13582 raft_consensus.cc:2258] T c1a791d491d64a308d0cf6466fffd5da P 2402c8bc50de4c2e93d2fa5e8ebc3a2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:09:11.394120 13581 raft_consensus.cc:2258] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:09:11.394718 13582 ts_tablet_manager.cc:1905] T c1a791d491d64a308d0cf6466fffd5da P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:09:11.394819 13581 ts_tablet_manager.cc:1905] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:09:11.397401 13581 ts_tablet_manager.cc:1918] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 20:09:11.397390 13582 ts_tablet_manager.cc:1918] T c1a791d491d64a308d0cf6466fffd5da P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 20:09:11.397483 13581 log.cc:1196] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/c1a791d491d64a308d0cf6466fffd5da
I20260620 20:09:11.397483 13582 log.cc:1196] T c1a791d491d64a308d0cf6466fffd5da P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-0/wal/wals/c1a791d491d64a308d0cf6466fffd5da
I20260620 20:09:11.397720 13581 ts_tablet_manager.cc:1939] T c1a791d491d64a308d0cf6466fffd5da P bceef69fdc22452ca520d306ac58656f: Deleting consensus metadata
I20260620 20:09:11.397720 13582 ts_tablet_manager.cc:1939] T c1a791d491d64a308d0cf6466fffd5da P 2402c8bc50de4c2e93d2fa5e8ebc3a2d: Deleting consensus metadata
I20260620 20:09:11.398427 13287 tablet_service.cc:1514] Processing DeleteTablet for tablet c1a791d491d64a308d0cf6466fffd5da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:09:11 UTC) from {username='oryx', principal='oryx/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:33930
I20260620 20:09:11.398612 13584 tablet_replica.cc:330] T c1a791d491d64a308d0cf6466fffd5da P c8a411d4fd0d48df8446d518e2d71a77: stopping tablet replica
I20260620 20:09:11.398727 13584 raft_consensus.cc:2229] T c1a791d491d64a308d0cf6466fffd5da P c8a411d4fd0d48df8446d518e2d71a77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:09:11.398721 12997 catalog_manager.cc:4841] TS bceef69fdc22452ca520d306ac58656f (127.5.91.131:33291): tablet c1a791d491d64a308d0cf6466fffd5da (table test-table [id=5bb97a9e43e9494a9b3ef91572499746]) successfully deleted
I20260620 20:09:11.398834 13584 raft_consensus.cc:2258] T c1a791d491d64a308d0cf6466fffd5da P c8a411d4fd0d48df8446d518e2d71a77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:09:11.399008 12994 catalog_manager.cc:4841] TS 2402c8bc50de4c2e93d2fa5e8ebc3a2d (127.5.91.129:43491): tablet c1a791d491d64a308d0cf6466fffd5da (table test-table [id=5bb97a9e43e9494a9b3ef91572499746]) successfully deleted
I20260620 20:09:11.399595 13584 ts_tablet_manager.cc:1905] T c1a791d491d64a308d0cf6466fffd5da P c8a411d4fd0d48df8446d518e2d71a77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:09:11.401608 13584 ts_tablet_manager.cc:1918] T c1a791d491d64a308d0cf6466fffd5da P c8a411d4fd0d48df8446d518e2d71a77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 20:09:11.401670 13584 log.cc:1196] T c1a791d491d64a308d0cf6466fffd5da P c8a411d4fd0d48df8446d518e2d71a77: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781986111135528-5486-0/minicluster-data/ts-1/wal/wals/c1a791d491d64a308d0cf6466fffd5da
I20260620 20:09:11.401854 13584 ts_tablet_manager.cc:1939] T c1a791d491d64a308d0cf6466fffd5da P c8a411d4fd0d48df8446d518e2d71a77: Deleting consensus metadata
I20260620 20:09:11.402525 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 13082
I20260620 20:09:11.405036 12994 catalog_manager.cc:4841] TS c8a411d4fd0d48df8446d518e2d71a77 (127.5.91.130:40845): tablet c1a791d491d64a308d0cf6466fffd5da (table test-table [id=5bb97a9e43e9494a9b3ef91572499746]) successfully deleted
I20260620 20:09:11.410022 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 13222
I20260620 20:09:11.415369 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 13362
I20260620 20:09:11.420593 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 12978
2026-06-20T20:09:11Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3740 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:09:11 dist-test-slave-l8h4 krb5kdc[13625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:09:11 dist-test-slave-l8h4 krb5kdc[13625](info): set up 2 sockets
Jun 20 20:09:11 dist-test-slave-l8h4 krb5kdc[13625](info): commencing operation
krb5kdc: starting...
W20260620 20:09:12.415664 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.973s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:09:12 dist-test-slave-l8h4 krb5kdc[13625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:09:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:09:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:09:12.547655 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:46333
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:44389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:46333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:09:12.670809 13638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:12.671110 13638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:12.671151 13638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:12.677057 13638 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:09:12.677134 13638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:12.677155 13638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:12.677170 13638 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:09:12.677184 13638 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:09:12.682971 13638 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:44389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:46333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:46333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:12.684082 13638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:12.685124 13638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:12.693789 13643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:12.693785 13644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:12.694034 13646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:12.695014 13638 server_base.cc:1034] running on GCE node
I20260620 20:09:12.695588 13638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:12.697113 13638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:12.698397 13638 hybrid_clock.cc:648] HybridClock initialized: now 1781986152698361 us; error 51 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-20T20:09:12Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1286 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:09:12 dist-test-slave-l8h4 krb5kdc[13651](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:09:12 dist-test-slave-l8h4 krb5kdc[13651](info): set up 2 sockets
Jun 20 20:09:12 dist-test-slave-l8h4 krb5kdc[13651](info): commencing operation
krb5kdc: starting...
W20260620 20:09:13.735874 5486 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:09:13 dist-test-slave-l8h4 krb5kdc[13651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:09:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:09:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:09:13.874883 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:33573
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:33573
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:09:14.011487 13664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:14.011838 13664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:14.012219 13664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:14.019510 13664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:09:14.019609 13664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:14.019631 13664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:14.019649 13664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:09:14.019665 13664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:09:14.027426 13664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:33573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:33573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:14.028985 13664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:14.030992 13664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:14.041501 13670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:14.041561 13664 server_base.cc:1034] running on GCE node
W20260620 20:09:14.041501 13669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:14.042049 13672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:14.042907 13664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:14.044987 13664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:14.046324 13664 hybrid_clock.cc:648] HybridClock initialized: now 1781986154046282 us; error 44 us; skew 500 ppm
Jun 20 20:09:14 dist-test-slave-l8h4 krb5kdc[13651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986154, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:14.052712 13664 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:09:14.053840 13664 webserver.cc:460] Webserver started at http://127.5.91.190:34431/ using document root <none> and password file <none>
I20260620 20:09:14.054618 13664 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:14.054718 13664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:14.054965 13664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:14.057204 13664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "edbe7500608e4159bf744a3a99c6aae4"
format_stamp: "Formatted at 2026-06-20 20:09:14 on dist-test-slave-l8h4"
server_key: "0f759d92a066005033d0dafe58a63ef0"
server_key_iv: "1e48d7cce9bc86baabd8e15ca6c60228"
server_key_version: "encryptionkey@0"
I20260620 20:09:14.057823 13664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "edbe7500608e4159bf744a3a99c6aae4"
format_stamp: "Formatted at 2026-06-20 20:09:14 on dist-test-slave-l8h4"
server_key: "0f759d92a066005033d0dafe58a63ef0"
server_key_iv: "1e48d7cce9bc86baabd8e15ca6c60228"
server_key_version: "encryptionkey@0"
I20260620 20:09:14.062067 13664 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260620 20:09:14.066224 13679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:14.068073 13664 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260620 20:09:14.068223 13664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "edbe7500608e4159bf744a3a99c6aae4"
format_stamp: "Formatted at 2026-06-20 20:09:14 on dist-test-slave-l8h4"
server_key: "0f759d92a066005033d0dafe58a63ef0"
server_key_iv: "1e48d7cce9bc86baabd8e15ca6c60228"
server_key_version: "encryptionkey@0"
I20260620 20:09:14.068351 13664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:14.083835 13664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:14.088220 13664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:14.088505 13664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:14.097949 13664 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:33573
I20260620 20:09:14.097968 13731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:33573 every 8 connection(s)
I20260620 20:09:14.099335 13664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:09:14.102968 13732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:14.103981 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 13664
I20260620 20:09:14.104111 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:09:14.104410 5486 external_mini_cluster.cc:1428] Setting key 255fb7b88a4c2a7a19faf0d4728c14da
I20260620 20:09:14.109568 13732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4: Bootstrap starting.
I20260620 20:09:14.112077 13732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:14.112862 13732 log.cc:824] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4: Log is configured to *not* fsync() on all Append() calls
Jun 20 20:09:14 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:09:14.115180 13732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4: No bootstrap required, opened a new log
I20260620 20:09:14.118824 13732 raft_consensus.cc:348] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbe7500608e4159bf744a3a99c6aae4" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33573 } }
I20260620 20:09:14.119053 13732 raft_consensus.cc:374] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:14.119102 13732 raft_consensus.cc:729] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edbe7500608e4159bf744a3a99c6aae4, State: Initialized, Role: FOLLOWER
I20260620 20:09:14.119531 13732 consensus_queue.cc:260] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbe7500608e4159bf744a3a99c6aae4" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33573 } }
I20260620 20:09:14.119657 13732 raft_consensus.cc:388] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:09:14.119721 13732 raft_consensus.cc:482] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:09:14.119813 13732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:14.120375 13735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:14.105965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40562 (local address 127.5.91.190:33573)
0620 20:09:14.106440 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:14.106456 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:14.106497 (+ 41us) server_negotiation.cc:407] Connection header received
0620 20:09:14.107683 (+ 1186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:14.107708 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:14.108190 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:14.108504 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:14.109471 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:14.110822 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:14.111980 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:14.112536 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:14.112652 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:14.115429 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:14.115438 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:14.115461 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:14.115519 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:14.117445 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:14.117872 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:14.117877 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:14.117882 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:14.118002 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:14.118285 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:14.118288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:14.118289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:14.118686 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:14.118885 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:14.119153 (+ 268us) server_negotiation.cc:299] Negotiation successful
0620 20:09:14.119552 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":157,"threads_started":1}
I20260620 20:09:14.120784 13732 raft_consensus.cc:504] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbe7500608e4159bf744a3a99c6aae4" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33573 } }
I20260620 20:09:14.121090 13732 leader_election.cc:302] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edbe7500608e4159bf744a3a99c6aae4; no voters:
I20260620 20:09:14.121378 13732 leader_election.cc:288] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:09:14.121452 13737 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:09:14.121661 13737 raft_consensus.cc:686] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [term 1 LEADER]: Becoming Leader. State: Replica: edbe7500608e4159bf744a3a99c6aae4, State: Running, Role: LEADER
I20260620 20:09:14.121989 13737 consensus_queue.cc:237] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbe7500608e4159bf744a3a99c6aae4" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33573 } }
I20260620 20:09:14.122713 13732 sys_catalog.cc:564] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:09:14.123155 13737 sys_catalog.cc:455] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader edbe7500608e4159bf744a3a99c6aae4. Latest consensus state: current_term: 1 leader_uuid: "edbe7500608e4159bf744a3a99c6aae4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbe7500608e4159bf744a3a99c6aae4" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33573 } } }
I20260620 20:09:14.123348 13737 sys_catalog.cc:458] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:14.123589 13738 sys_catalog.cc:455] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edbe7500608e4159bf744a3a99c6aae4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbe7500608e4159bf744a3a99c6aae4" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33573 } } }
I20260620 20:09:14.123752 13738 sys_catalog.cc:458] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:14.123762 13743 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:09:14.127013 13743 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:09:14.132625 13743 catalog_manager.cc:1347] Generated new cluster ID: 6caf55aece4544f8aceee3183ebe73ed
I20260620 20:09:14.132704 13743 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:09:14.172539 13743 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:09:14.173514 13743 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:09:14.185835 13743 catalog_manager.cc:5868] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4: Generated new TSK 0
I20260620 20:09:14.186620 13743 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.91.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
I20260620 20:09:14.229738 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:33573
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:09:14.356257 13758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:14.356603 13758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:14.356657 13758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:14.360896 13758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:14.360961 13758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:14.361022 13758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:09:14.366621 13758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:33573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:14.367733 13758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:14.368734 13758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 20:09:14.378911 13758 server_base.cc:1034] running on GCE node
W20260620 20:09:14.378959 13764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:14.378882 13766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:14.378882 13763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:14.379647 13758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:14.380295 13758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:14.381517 13758 hybrid_clock.cc:648] HybridClock initialized: now 1781986154381491 us; error 40 us; skew 500 ppm
Jun 20 20:09:14 dist-test-slave-l8h4 krb5kdc[13651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986154, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:14.384527 13758 init.cc:377] Logged in from keytab as kudu/127.5.91.129@KRBTEST.COM (short username kudu)
I20260620 20:09:14.385453 13758 webserver.cc:460] Webserver started at http://127.5.91.129:41425/ using document root <none> and password file <none>
I20260620 20:09:14.385983 13758 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:14.386044 13758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:14.386277 13758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:14.388371 13758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "44b1fa32487f49bd817f42f0dbe6b9d1"
format_stamp: "Formatted at 2026-06-20 20:09:14 on dist-test-slave-l8h4"
server_key: "620016cf07e348c5b44a0ff758c34abb"
server_key_iv: "767b78060a17e398f087c2e9ca4bb0b7"
server_key_version: "encryptionkey@0"
I20260620 20:09:14.389129 13758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "44b1fa32487f49bd817f42f0dbe6b9d1"
format_stamp: "Formatted at 2026-06-20 20:09:14 on dist-test-slave-l8h4"
server_key: "620016cf07e348c5b44a0ff758c34abb"
server_key_iv: "767b78060a17e398f087c2e9ca4bb0b7"
server_key_version: "encryptionkey@0"
I20260620 20:09:14.393566 13758 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260620 20:09:14.396777 13773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:14.398214 13758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:09:14.398331 13758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "44b1fa32487f49bd817f42f0dbe6b9d1"
format_stamp: "Formatted at 2026-06-20 20:09:14 on dist-test-slave-l8h4"
server_key: "620016cf07e348c5b44a0ff758c34abb"
server_key_iv: "767b78060a17e398f087c2e9ca4bb0b7"
server_key_version: "encryptionkey@0"
I20260620 20:09:14.398447 13758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:14.439075 13758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:14.443087 13758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:14.443286 13758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:14.443884 13758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:14.444708 13758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:14.444756 13758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:14.444809 13758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:14.444834 13758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:14.455796 13758 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:34251
I20260620 20:09:14.455853 13886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:34251 every 8 connection(s)
I20260620 20:09:14.457438 13758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:09:14.458509 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 13758
I20260620 20:09:14.458619 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:09:14.458864 5486 external_mini_cluster.cc:1428] Setting key 482a3ce52dc962ef9e6025dd72e96091
WARNING: no policy specified for kudu/127.5.91.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.130@KRBTEST.COM" created.
Jun 20 20:09:14 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986154, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:09:14.478641 13735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:14.460432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:45695 (local address 127.5.91.190:33573)
0620 20:09:14.460585 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:14.460590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:14.461814 (+ 1224us) server_negotiation.cc:407] Connection header received
0620 20:09:14.463587 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:14.463590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:14.463647 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:14.463736 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:14.465048 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:14.465618 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:14.466809 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:14.467048 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:14.467137 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:14.473452 (+ 6315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:14.473457 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:14.473459 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:14.473496 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:14.476126 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:14.476757 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:14.476761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:14.476763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:14.476857 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:14.477277 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:14.477280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:14.477282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:14.477532 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:14.477631 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:14.478284 (+ 653us) server_negotiation.cc:299] Negotiation successful
0620 20:09:14.478424 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260620 20:09:14.479529 13889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:14.460873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.129:45695)
0620 20:09:14.461661 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:14.461713 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:14.463272 (+ 1559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:14.463966 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:14.463973 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:14.464364 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:14.464882 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:14.464897 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:14.465744 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:14.465747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:14.466685 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:14.466696 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:14.467191 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:14.467195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:14.467359 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:14.467962 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:14.467985 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:14.472848 (+ 4863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:14.476284 (+ 3436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:14.476289 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:14.476306 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:14.476635 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:14.477013 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:14.477017 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:14.477019 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:14.477173 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:14.477652 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:14.477659 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:14.478004 (+ 345us) client_negotiation.cc:769] Sending connection context
0620 20:09:14.478221 (+ 217us) client_negotiation.cc:240] Negotiation successful
0620 20:09:14.478645 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":116,"threads_started":1}
I20260620 20:09:14.480921 13887 heartbeater.cc:347] Connected to a master server at 127.5.91.190:33573
I20260620 20:09:14.481148 13887 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:14.481721 13887 heartbeater.cc:511] Master 127.5.91.190:33573 requested a full tablet report, sending...
I20260620 20:09:14.483780 13690 ts_manager.cc:194] Registered new tserver with Master: 44b1fa32487f49bd817f42f0dbe6b9d1 (127.5.91.129:34251)
I20260620 20:09:14.484916 13690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:45695
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
I20260620 20:09:14.493489 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:33573
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:09:14.606209 13892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:14.606493 13892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:14.606537 13892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:14.610756 13892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:14.610888 13892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:14.610953 13892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:09:14.616456 13892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:33573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:14.617522 13892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:14.618824 13892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:14.629472 13900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:14.629622 13892 server_base.cc:1034] running on GCE node
W20260620 20:09:14.629453 13897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:14.629423 13898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:14.630169 13892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:14.630853 13892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:14.632045 13892 hybrid_clock.cc:648] HybridClock initialized: now 1781986154632007 us; error 49 us; skew 500 ppm
Jun 20 20:09:14 dist-test-slave-l8h4 krb5kdc[13651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986154, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:14.635119 13892 init.cc:377] Logged in from keytab as kudu/127.5.91.130@KRBTEST.COM (short username kudu)
I20260620 20:09:14.636049 13892 webserver.cc:460] Webserver started at http://127.5.91.130:33111/ using document root <none> and password file <none>
I20260620 20:09:14.636588 13892 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:14.636653 13892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:14.636869 13892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:14.638964 13892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "d17d7cfea54041229de573ea27eae273"
format_stamp: "Formatted at 2026-06-20 20:09:14 on dist-test-slave-l8h4"
server_key: "d0185df74fdb21d494eac2cbba663beb"
server_key_iv: "59f53345af8621c2f029f724274b29e9"
server_key_version: "encryptionkey@0"
I20260620 20:09:14.639524 13892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "d17d7cfea54041229de573ea27eae273"
format_stamp: "Formatted at 2026-06-20 20:09:14 on dist-test-slave-l8h4"
server_key: "d0185df74fdb21d494eac2cbba663beb"
server_key_iv: "59f53345af8621c2f029f724274b29e9"
server_key_version: "encryptionkey@0"
I20260620 20:09:14.643294 13892 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260620 20:09:14.646250 13907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:14.647575 13892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:09:14.647692 13892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "d17d7cfea54041229de573ea27eae273"
format_stamp: "Formatted at 2026-06-20 20:09:14 on dist-test-slave-l8h4"
server_key: "d0185df74fdb21d494eac2cbba663beb"
server_key_iv: "59f53345af8621c2f029f724274b29e9"
server_key_version: "encryptionkey@0"
I20260620 20:09:14.647799 13892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:14.670907 13892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:14.674373 13892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:14.674574 13892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:14.675118 13892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:14.676146 13892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:14.676221 13892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:14.676268 13892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:14.676283 13892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:14.686578 13892 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:41613
I20260620 20:09:14.686614 14020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:41613 every 8 connection(s)
I20260620 20:09:14.687983 13892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:09:14.690555 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 13892
I20260620 20:09:14.690663 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:09:14.690935 5486 external_mini_cluster.cc:1428] Setting key fa3277dd65f10bfebec0e8e1904c11c1
Jun 20 20:09:14 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986154, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.91.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.131@KRBTEST.COM" created.
I20260620 20:09:14.707108 13735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:14.690242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:45267 (local address 127.5.91.190:33573)
0620 20:09:14.690429 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:14.690433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:14.691839 (+ 1406us) server_negotiation.cc:407] Connection header received
0620 20:09:14.692854 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:14.692857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:14.692907 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:14.692978 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:14.694156 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:14.695162 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:14.697365 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:14.697641 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:14.697842 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:14.701562 (+ 3720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:14.701566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:14.701568 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:14.701593 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:14.703202 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:14.704127 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:14.704133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:14.704134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:14.704337 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:14.704940 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:14.704943 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:14.704944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:14.705161 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:14.705258 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:14.706497 (+ 1239us) server_negotiation.cc:299] Negotiation successful
0620 20:09:14.706875 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 20:09:14.708194 14023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:14.690651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.130:45267)
0620 20:09:14.691685 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:14.691742 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:14.692657 (+ 915us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:14.693143 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:14.693150 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:14.693498 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:14.693993 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:14.694007 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:14.695455 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:14.695459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:14.697120 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:14.697144 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:14.697945 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:14.697951 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:14.698404 (+ 453us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:14.699095 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:14.699118 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:14.701312 (+ 2194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:14.703391 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:14.703402 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:14.703431 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:14.703937 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:14.704511 (+ 574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:14.704514 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:14.704517 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:14.704793 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:14.705345 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:14.705354 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:14.705917 (+ 563us) client_negotiation.cc:769] Sending connection context
0620 20:09:14.706379 (+ 462us) client_negotiation.cc:240] Negotiation successful
0620 20:09:14.707004 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":821,"thread_start_us":539,"threads_started":1}
I20260620 20:09:14.710773 14021 heartbeater.cc:347] Connected to a master server at 127.5.91.190:33573
I20260620 20:09:14.711114 14021 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:14.711767 14021 heartbeater.cc:511] Master 127.5.91.190:33573 requested a full tablet report, sending...
I20260620 20:09:14.713335 13690 ts_manager.cc:194] Registered new tserver with Master: d17d7cfea54041229de573ea27eae273 (127.5.91.130:41613)
I20260620 20:09:14.714505 13690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:45267
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
I20260620 20:09:14.730031 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:33573
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:09:14.845033 14026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:14.845315 14026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:14.845359 14026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:14.849956 14026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:14.850034 14026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:14.850111 14026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:09:14.855703 14026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:33573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:14.856738 14026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:14.858492 14026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:14.870043 14031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:14.870059 14034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:14.870059 14032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:14.870319 14026 server_base.cc:1034] running on GCE node
I20260620 20:09:14.870828 14026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:14.871480 14026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:14.872669 14026 hybrid_clock.cc:648] HybridClock initialized: now 1781986154872619 us; error 79 us; skew 500 ppm
Jun 20 20:09:14 dist-test-slave-l8h4 krb5kdc[13651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986154, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:14.875501 14026 init.cc:377] Logged in from keytab as kudu/127.5.91.131@KRBTEST.COM (short username kudu)
I20260620 20:09:14.876368 14026 webserver.cc:460] Webserver started at http://127.5.91.131:40403/ using document root <none> and password file <none>
I20260620 20:09:14.876914 14026 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:14.876983 14026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:14.877177 14026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:14.879282 14026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "09708e7b6c5a49658e28b4697ac2ae45"
format_stamp: "Formatted at 2026-06-20 20:09:14 on dist-test-slave-l8h4"
server_key: "55224aed2474da838f9300c7c0ddae53"
server_key_iv: "ba9c7948e4aba338a47419de85ea5130"
server_key_version: "encryptionkey@0"
I20260620 20:09:14.879805 14026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "09708e7b6c5a49658e28b4697ac2ae45"
format_stamp: "Formatted at 2026-06-20 20:09:14 on dist-test-slave-l8h4"
server_key: "55224aed2474da838f9300c7c0ddae53"
server_key_iv: "ba9c7948e4aba338a47419de85ea5130"
server_key_version: "encryptionkey@0"
I20260620 20:09:14.883728 14026 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 20:09:14.886204 14041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:14.887468 14026 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:09:14.887581 14026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "09708e7b6c5a49658e28b4697ac2ae45"
format_stamp: "Formatted at 2026-06-20 20:09:14 on dist-test-slave-l8h4"
server_key: "55224aed2474da838f9300c7c0ddae53"
server_key_iv: "ba9c7948e4aba338a47419de85ea5130"
server_key_version: "encryptionkey@0"
I20260620 20:09:14.887684 14026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:14.902546 14026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:14.906605 14026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:14.906802 14026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:14.907383 14026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:14.908234 14026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:14.908282 14026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:14.908334 14026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:14.908363 14026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:14.917917 14026 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:39199
I20260620 20:09:14.918000 14154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:39199 every 8 connection(s)
I20260620 20:09:14.919066 14026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:09:14.926736 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 14026
I20260620 20:09:14.926918 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:09:14.927285 5486 external_mini_cluster.cc:1428] Setting key 7f0860c70e5ef0a9a5b92aedeaf78479
Jun 20 20:09:14 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986154, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:09:14.935523 13735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:14.921093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:52167 (local address 127.5.91.190:33573)
0620 20:09:14.921222 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:14.921226 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:14.922349 (+ 1123us) server_negotiation.cc:407] Connection header received
0620 20:09:14.923298 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:14.923301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:14.923350 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:14.923428 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:14.924744 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:14.925424 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:14.926884 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:14.927183 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:14.927315 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:14.930692 (+ 3377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:14.930696 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:14.930698 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:14.930723 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:14.932301 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:14.933279 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:14.933283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:14.933285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:14.933390 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:14.933997 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:14.934000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:14.934002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:14.934262 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:14.934353 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:14.935258 (+ 905us) server_negotiation.cc:299] Negotiation successful
0620 20:09:14.935384 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 20:09:14.936380 14157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:14.921409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.131:52167)
0620 20:09:14.922107 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:14.922193 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:14.923060 (+ 867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:14.923600 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:14.923607 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:14.924064 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:14.924578 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:14.924592 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:14.925750 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:14.925754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:14.926728 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:14.926738 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:14.927472 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:14.927479 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:14.927884 (+ 405us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:14.928707 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:14.928775 (+ 68us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:14.930379 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:14.932607 (+ 2228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:14.932615 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:14.932638 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:14.933071 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:14.933548 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:14.933551 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:14.933553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:14.933742 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:14.934392 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:14.934398 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:14.934909 (+ 511us) client_negotiation.cc:769] Sending connection context
0620 20:09:14.935171 (+ 262us) client_negotiation.cc:240] Negotiation successful
0620 20:09:14.935515 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":118,"threads_started":1}
I20260620 20:09:14.937569 14155 heartbeater.cc:347] Connected to a master server at 127.5.91.190:33573
I20260620 20:09:14.937785 14155 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:14.938283 14155 heartbeater.cc:511] Master 127.5.91.190:33573 requested a full tablet report, sending...
I20260620 20:09:14.939359 13690 ts_manager.cc:194] Registered new tserver with Master: 09708e7b6c5a49658e28b4697ac2ae45 (127.5.91.131:39199)
I20260620 20:09:14.939994 13690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:52167
I20260620 20:09:14.941805 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.91.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.91.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.91.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.129.keytab.
Entry for principal oryx/127.5.91.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.129.keytab.
WARNING: no policy specified for oryx/127.5.91.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.91.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.91.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.130.keytab.
Entry for principal oryx/127.5.91.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.130.keytab.
WARNING: no policy specified for oryx/127.5.91.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.91.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.91.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.131.keytab.
Entry for principal oryx/127.5.91.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.131.keytab.
I20260620 20:09:15.051230 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 13758
I20260620 20:09:15.057792 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 13892
I20260620 20:09:15.063361 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 14026
I20260620 20:09:15.069469 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 13664
I20260620 20:09:15.076380 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:33573
--webserver_interface=127.5.91.190
--webserver_port=34431
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:33573
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:09:15.194695 14164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:15.194976 14164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:15.195034 14164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:15.199339 14164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:09:15.199414 14164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:15.199446 14164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:15.199468 14164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:09:15.199489 14164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:09:15.205092 14164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:33573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:33573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=34431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:15.206234 14164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:15.207255 14164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:15.216213 14172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:15.216286 14164 server_base.cc:1034] running on GCE node
W20260620 20:09:15.216179 14169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:15.216179 14170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:15.217195 14164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:15.218365 14164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:15.219699 14164 hybrid_clock.cc:648] HybridClock initialized: now 1781986155219676 us; error 59 us; skew 500 ppm
Jun 20 20:09:15 dist-test-slave-l8h4 krb5kdc[13651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986155, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:15.224071 14164 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:09:15.225247 14164 webserver.cc:460] Webserver started at http://127.5.91.190:34431/ using document root <none> and password file <none>
I20260620 20:09:15.225809 14164 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:15.225940 14164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:15.231308 14164 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 20:09:15.233831 14179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:15.235445 14164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 20:09:15.235590 14164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "edbe7500608e4159bf744a3a99c6aae4"
format_stamp: "Formatted at 2026-06-20 20:09:14 on dist-test-slave-l8h4"
server_key: "0f759d92a066005033d0dafe58a63ef0"
server_key_iv: "1e48d7cce9bc86baabd8e15ca6c60228"
server_key_version: "encryptionkey@0"
I20260620 20:09:15.236191 14164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:15.268187 14164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:15.272189 14164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:15.272393 14164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:15.279671 14164 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:33573
I20260620 20:09:15.279728 14231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:33573 every 8 connection(s)
I20260620 20:09:15.280756 14164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:09:15.284444 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 14164
I20260620 20:09:15.284898 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:34251
--local_ip_for_outbound_sockets=127.5.91.129
--tserver_master_addrs=127.5.91.190:33573
--webserver_port=41425
--webserver_interface=127.5.91.129
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.91.129
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 20:09:15.285517 14232 sys_catalog.cc:263] Verifying existing consensus state
I20260620 20:09:15.287194 14232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4: Bootstrap starting.
I20260620 20:09:15.308769 14232 log.cc:824] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4: Log is configured to *not* fsync() on all Append() calls
I20260620 20:09:15.316236 14232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 20:09:15.316743 14232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4: Bootstrap complete.
I20260620 20:09:15.321441 14232 raft_consensus.cc:348] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbe7500608e4159bf744a3a99c6aae4" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33573 } }
I20260620 20:09:15.321889 14232 raft_consensus.cc:729] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: edbe7500608e4159bf744a3a99c6aae4, State: Initialized, Role: FOLLOWER
I20260620 20:09:15.322419 14232 consensus_queue.cc:260] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbe7500608e4159bf744a3a99c6aae4" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33573 } }
I20260620 20:09:15.322539 14232 raft_consensus.cc:388] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:09:15.322602 14232 raft_consensus.cc:482] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:09:15.322700 14232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [term 1 FOLLOWER]: Advancing to term 2
I20260620 20:09:15.324658 14232 raft_consensus.cc:504] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbe7500608e4159bf744a3a99c6aae4" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33573 } }
I20260620 20:09:15.324991 14232 leader_election.cc:302] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edbe7500608e4159bf744a3a99c6aae4; no voters:
I20260620 20:09:15.325292 14232 leader_election.cc:288] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 20:09:15.325553 14236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 20:09:15.325928 14236 raft_consensus.cc:686] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [term 2 LEADER]: Becoming Leader. State: Replica: edbe7500608e4159bf744a3a99c6aae4, State: Running, Role: LEADER
I20260620 20:09:15.326457 14236 consensus_queue.cc:237] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbe7500608e4159bf744a3a99c6aae4" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33573 } }
I20260620 20:09:15.326599 14232 sys_catalog.cc:564] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:09:15.327952 14237 sys_catalog.cc:455] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "edbe7500608e4159bf744a3a99c6aae4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbe7500608e4159bf744a3a99c6aae4" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33573 } } }
I20260620 20:09:15.328087 14237 sys_catalog.cc:458] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:15.328413 14238 sys_catalog.cc:455] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader edbe7500608e4159bf744a3a99c6aae4. Latest consensus state: current_term: 2 leader_uuid: "edbe7500608e4159bf744a3a99c6aae4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbe7500608e4159bf744a3a99c6aae4" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33573 } } }
I20260620 20:09:15.328495 14238 sys_catalog.cc:458] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4 [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:15.331920 14251 catalog_manager.cc:1259] Loaded cluster ID: 6caf55aece4544f8aceee3183ebe73ed
I20260620 20:09:15.331987 14251 catalog_manager.cc:1552] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4: loading cluster ID for follower catalog manager: success
I20260620 20:09:15.334514 14251 catalog_manager.cc:1574] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4: acquiring CA information for follower catalog manager: success
I20260620 20:09:15.336370 14251 catalog_manager.cc:1602] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 20:09:15.336545 14246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:09:15.337488 14246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:09:15.337952 14246 catalog_manager.cc:1259] Loaded cluster ID: 6caf55aece4544f8aceee3183ebe73ed
I20260620 20:09:15.338014 14246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:09:15.338706 14246 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:09:15.339171 14246 catalog_manager.cc:5879] T 00000000000000000000000000000000 P edbe7500608e4159bf744a3a99c6aae4: Loaded TSK: 0
I20260620 20:09:15.339977 14246 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 20:09:15.415483 14234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:15.415756 14234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:15.415800 14234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:15.421628 14234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:15.421710 14234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:15.421792 14234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:09:15.428256 14234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.129.keytab
--principal=oryx/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:34251
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=41425
--tserver_master_addrs=127.5.91.190:33573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:15.429311 14234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:15.430351 14234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:15.442417 14258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:15.442523 14259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:15.442440 14261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:15.443045 14234 server_base.cc:1034] running on GCE node
I20260620 20:09:15.443648 14234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:15.444550 14234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:15.445773 14234 hybrid_clock.cc:648] HybridClock initialized: now 1781986155445742 us; error 41 us; skew 500 ppm
Jun 20 20:09:15 dist-test-slave-l8h4 krb5kdc[13651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986155, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:15.449472 14234 init.cc:377] Logged in from keytab as oryx/127.5.91.129@KRBTEST.COM (short username oryx)
I20260620 20:09:15.450515 14234 webserver.cc:460] Webserver started at http://127.5.91.129:41425/ using document root <none> and password file <none>
I20260620 20:09:15.451076 14234 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:15.451146 14234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:15.457083 14234 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 20:09:15.460531 14268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:15.462574 14234 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260620 20:09:15.462774 14234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "44b1fa32487f49bd817f42f0dbe6b9d1"
format_stamp: "Formatted at 2026-06-20 20:09:14 on dist-test-slave-l8h4"
server_key: "620016cf07e348c5b44a0ff758c34abb"
server_key_iv: "767b78060a17e398f087c2e9ca4bb0b7"
server_key_version: "encryptionkey@0"
I20260620 20:09:15.463608 14234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:15.477305 14234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:15.480572 14234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:15.480751 14234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:15.481330 14234 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:15.482239 14234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:15.482285 14234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:15.482339 14234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:15.482368 14234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:15.491379 14234 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:34251
I20260620 20:09:15.491457 14381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:34251 every 8 connection(s)
I20260620 20:09:15.492434 14234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:09:15.502473 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 14234
Jun 20 20:09:15 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:15.503193 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:41613
--local_ip_for_outbound_sockets=127.5.91.130
--tserver_master_addrs=127.5.91.190:33573
--webserver_port=33111
--webserver_interface=127.5.91.130
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.91.130
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 20:09:15 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
W20260620 20:09:15.506036 14384 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:15.494606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:52341 (local address 127.5.91.190:33573)
0620 20:09:15.495191 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:15.495218 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:15.495617 (+ 399us) server_negotiation.cc:407] Connection header received
0620 20:09:15.496692 (+ 1075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:15.496702 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:15.497031 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:15.497318 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:15.498470 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.499413 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.500507 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.500957 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.501109 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:15.505290 (+ 4181us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.129:52341: BlockingRecv error: recv got EOF from 127.5.91.129:52341 (error 108)
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":134,"threads_started":1}
I20260620 20:09:15.506045 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:15.494760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.129:52341)
0620 20:09:15.495463 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:15.495503 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:15.496312 (+ 809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:15.497438 (+ 1126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:15.497446 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:15.497830 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:15.498304 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:15.498320 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.499540 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.499543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:15.500380 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:15.500389 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.501107 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.501111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:15.501277 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:15.502039 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:15.502057 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:15.505028 (+ 2971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":136,"threads_started":1}
W20260620 20:09:15.506737 14382 heartbeater.cc:644] Failed to heartbeat to 127.5.91.190:33573 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.91.190:33573: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Jun 20 20:09:15 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:15 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:15.513479 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:15.507155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.129:34117)
0620 20:09:15.507325 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:15.507338 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:15.507421 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:15.507870 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:15.507872 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:15.508023 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:15.508123 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:15.508128 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.509263 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.509265 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:15.509823 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:15.509831 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.510432 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.510435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:15.510517 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:15.511002 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:15.511014 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:15.513336 (+ 2322us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101}
W20260620 20:09:15.513651 14384 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:15.507256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:34117 (local address 127.5.91.190:33573)
0620 20:09:15.507404 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:15.507408 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:15.507424 (+ 16us) server_negotiation.cc:407] Connection header received
0620 20:09:15.507631 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:15.507635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:15.507699 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:15.507796 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:15.508252 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.509143 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.509936 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.510292 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.510414 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:15.513528 (+ 3114us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.129:34117: BlockingRecv error: recv got EOF from 127.5.91.129:34117 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 20 20:09:15 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:15 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:15.520896 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:15.514192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.129:50445)
0620 20:09:15.514497 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:15.514511 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:15.514604 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:15.514868 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:15.514871 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:15.515074 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:15.515163 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:15.515169 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.516300 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.516305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:15.517036 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:15.517053 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.517797 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.517801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:15.517887 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:15.518625 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:15.518639 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:15.520788 (+ 2149us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234}
W20260620 20:09:15.521070 14384 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:15.514263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:50445 (local address 127.5.91.190:33573)
0620 20:09:15.514405 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:15.514409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:15.514573 (+ 164us) server_negotiation.cc:407] Connection header received
0620 20:09:15.514696 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:15.514699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:15.514748 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:15.514825 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:15.515259 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.516065 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.517258 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.517623 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.517734 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:15.520954 (+ 3220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.129:50445: BlockingRecv error: recv got EOF from 127.5.91.129:50445 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260620 20:09:15.521138 14382 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 20:09:15.623082 14386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:15.623390 14386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:15.623440 14386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:15.629587 14386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:15.629662 14386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:15.629727 14386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:09:15.636837 14386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.130.keytab
--principal=oryx/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:41613
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=33111
--tserver_master_addrs=127.5.91.190:33573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:15.638010 14386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:15.639202 14386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:15.649526 14394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:15.649537 14392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:15.649537 14391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:15.649745 14386 server_base.cc:1034] running on GCE node
I20260620 20:09:15.650373 14386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:15.651093 14386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:15.652282 14386 hybrid_clock.cc:648] HybridClock initialized: now 1781986155652232 us; error 68 us; skew 500 ppm
Jun 20 20:09:15 dist-test-slave-l8h4 krb5kdc[13651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986155, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:15.655488 14386 init.cc:377] Logged in from keytab as oryx/127.5.91.130@KRBTEST.COM (short username oryx)
I20260620 20:09:15.656412 14386 webserver.cc:460] Webserver started at http://127.5.91.130:33111/ using document root <none> and password file <none>
I20260620 20:09:15.656985 14386 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:15.657073 14386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:15.661309 14386 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 20:09:15.663535 14401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:15.665089 14386 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 20:09:15.665212 14386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "d17d7cfea54041229de573ea27eae273"
format_stamp: "Formatted at 2026-06-20 20:09:14 on dist-test-slave-l8h4"
server_key: "d0185df74fdb21d494eac2cbba663beb"
server_key_iv: "59f53345af8621c2f029f724274b29e9"
server_key_version: "encryptionkey@0"
I20260620 20:09:15.665902 14386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:15.683454 14386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:15.687376 14386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:15.687775 14386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:15.688534 14386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:15.689466 14386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:15.689517 14386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:15.689555 14386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:15.689582 14386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:15.699554 14386 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:41613
I20260620 20:09:15.699615 14514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:41613 every 8 connection(s)
I20260620 20:09:15.700589 14386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:09:15.710505 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 14386
I20260620 20:09:15.710925 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:39199
--local_ip_for_outbound_sockets=127.5.91.131
--tserver_master_addrs=127.5.91.190:33573
--webserver_port=40403
--webserver_interface=127.5.91.131
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.91.131
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 20:09:15 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:15 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
W20260620 20:09:15.713789 14384 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:15.702579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:57841 (local address 127.5.91.190:33573)
0620 20:09:15.702886 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:15.702890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:15.704126 (+ 1236us) server_negotiation.cc:407] Connection header received
0620 20:09:15.705558 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:15.705561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:15.705626 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:15.705698 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:15.707205 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.708137 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.709333 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.709548 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.709637 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:15.713640 (+ 4003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.130:57841: BlockingRecv error: recv got EOF from 127.5.91.130:57841 (error 108)
Metrics: {"server-negotiator.queue_time_us":211}
I20260620 20:09:15.714435 14517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:15.702883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.130:57841)
0620 20:09:15.703743 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:15.703818 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:15.705303 (+ 1485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:15.705907 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:15.705914 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:15.706328 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:15.706962 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:15.706979 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.708288 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.708291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:15.709200 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:15.709211 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.709700 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.709704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:15.709875 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:15.710734 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:15.710752 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:15.713399 (+ 2647us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":120,"threads_started":1}
W20260620 20:09:15.714998 14515 heartbeater.cc:644] Failed to heartbeat to 127.5.91.190:33573 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.91.190:33573: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Jun 20 20:09:15 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:15 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:15.721786 14517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:15.715510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.130:50543)
0620 20:09:15.715624 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:15.715638 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:15.715728 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:15.716156 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:15.716160 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:15.716370 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:15.716538 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:15.716545 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.717686 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.717690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:15.718492 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:15.718504 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.719373 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.719377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:15.719462 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:15.720003 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:15.720015 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:15.721675 (+ 1660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260620 20:09:15.722028 14384 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:15.715571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:50543 (local address 127.5.91.190:33573)
0620 20:09:15.715695 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:15.715699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:15.715716 (+ 17us) server_negotiation.cc:407] Connection header received
0620 20:09:15.715821 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:15.715825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:15.715875 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:15.715971 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:15.716718 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.717517 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.718691 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.719155 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.719362 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:15.721886 (+ 2524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.130:50543: BlockingRecv error: recv got EOF from 127.5.91.130:50543 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 20 20:09:15 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:15 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:15.728993 14517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:15.722451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.130:50533)
0620 20:09:15.722543 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:15.722557 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:15.722625 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:15.722971 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:15.722976 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:15.723203 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:15.723343 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:15.723350 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.724558 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.724561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:15.725201 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:15.725211 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.725762 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.725765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:15.725843 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:15.726334 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:15.726346 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:15.728814 (+ 2468us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260620 20:09:15.729398 14515 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 20:09:15.729251 14384 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:15.722526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:50533 (local address 127.5.91.190:33573)
0620 20:09:15.722639 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:15.722643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:15.722658 (+ 15us) server_negotiation.cc:407] Connection header received
0620 20:09:15.722703 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:15.722706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:15.722752 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:15.722841 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:15.723591 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.724421 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.725432 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.725661 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.725751 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:15.728994 (+ 3243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.130:50533: BlockingRecv error: recv got EOF from 127.5.91.130:50533 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260620 20:09:15.839864 14518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:15.840209 14518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:15.840291 14518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:15.845556 14518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:15.845633 14518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:15.845697 14518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:09:15.853148 14518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/krb5kdc/oryx_127.5.91.131.keytab
--principal=oryx/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:39199
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=40403
--tserver_master_addrs=127.5.91.190:33573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:15.854702 14518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:15.855770 14518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:15.871459 14523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:15.871537 14524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:15.871534 14526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:15.871994 14518 server_base.cc:1034] running on GCE node
I20260620 20:09:15.872488 14518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:15.873778 14518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:15.875291 14518 hybrid_clock.cc:648] HybridClock initialized: now 1781986155875238 us; error 59 us; skew 500 ppm
Jun 20 20:09:15 dist-test-slave-l8h4 krb5kdc[13651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986155, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:15.881723 14518 init.cc:377] Logged in from keytab as oryx/127.5.91.131@KRBTEST.COM (short username oryx)
I20260620 20:09:15.883113 14518 webserver.cc:460] Webserver started at http://127.5.91.131:40403/ using document root <none> and password file <none>
I20260620 20:09:15.883720 14518 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:15.883801 14518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:15.888671 14518 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260620 20:09:15.892530 14533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:15.894554 14518 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260620 20:09:15.894692 14518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "09708e7b6c5a49658e28b4697ac2ae45"
format_stamp: "Formatted at 2026-06-20 20:09:14 on dist-test-slave-l8h4"
server_key: "55224aed2474da838f9300c7c0ddae53"
server_key_iv: "ba9c7948e4aba338a47419de85ea5130"
server_key_version: "encryptionkey@0"
I20260620 20:09:15.895931 14518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:15.930663 14518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:15.934888 14518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:15.935083 14518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:15.935936 14518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:15.936962 14518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:15.937018 14518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:15.937059 14518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:15.937074 14518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:15.949728 14518 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:39199
I20260620 20:09:15.949790 14646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:39199 every 8 connection(s)
I20260620 20:09:15.951057 14518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:09:15.953259 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 14518
Jun 20 20:09:15 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:15 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
W20260620 20:09:15.963011 14384 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:15.953291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:54413 (local address 127.5.91.190:33573)
0620 20:09:15.953727 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:15.953732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:15.954584 (+ 852us) server_negotiation.cc:407] Connection header received
0620 20:09:15.955578 (+ 994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:15.955581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:15.955622 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:15.955687 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:15.956774 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.957275 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.958682 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.958900 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.958981 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:15.962844 (+ 3863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.131:54413: BlockingRecv error: recv got EOF from 127.5.91.131:54413 (error 108)
Metrics: {"server-negotiator.queue_time_us":334}
I20260620 20:09:15.963461 14651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:15.954617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40578 (local address 127.5.91.190:33573)
0620 20:09:15.954819 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:15.954823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:15.954834 (+ 11us) server_negotiation.cc:407] Connection header received
0620 20:09:15.954873 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:15.954875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:15.954919 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:15.955064 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:15.955798 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.956337 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.957184 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.957407 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.957490 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:15.959414 (+ 1924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:15.959422 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:15.959446 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:15.959495 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:15.961358 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:15.961792 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:15.961798 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:15.961803 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:15.961926 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:15.962211 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:15.962215 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:15.962217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:15.962556 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:15.962737 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:15.963042 (+ 305us) server_negotiation.cc:299] Negotiation successful
0620 20:09:15.963282 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":56,"threads_started":1}
I20260620 20:09:15.963908 14649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:15.953641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.131:54413)
0620 20:09:15.954445 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:15.954495 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:15.955387 (+ 892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:15.955846 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:15.955852 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:15.956177 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:15.956641 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:15.956654 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.957391 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.957393 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:15.958563 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:15.958574 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.959037 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.959042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:15.959188 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:15.959679 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:15.959696 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:15.962622 (+ 2926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":149,"threads_started":1}
W20260620 20:09:15.964478 14647 heartbeater.cc:644] Failed to heartbeat to 127.5.91.190:33573 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.91.190:33573: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Jun 20 20:09:15 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:15 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:15.972391 14649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:15.965005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.131:48711)
0620 20:09:15.965305 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:15.965323 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:15.965466 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:15.966020 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:15.966023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:15.966381 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:15.966506 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:15.966514 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.967700 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.967703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:15.968535 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:15.968555 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.969375 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.969378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:15.969478 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:15.969996 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:15.970007 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:15.972264 (+ 2257us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199}
W20260620 20:09:15.973115 14651 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:15.965136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:48711 (local address 127.5.91.190:33573)
0620 20:09:15.965318 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:15.965322 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:15.965473 (+ 151us) server_negotiation.cc:407] Connection header received
0620 20:09:15.965553 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:15.965556 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:15.965606 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:15.965803 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:15.966658 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.967525 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.968805 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.969111 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.969219 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:15.972817 (+ 3598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.131:48711: BlockingRecv error: recv got EOF from 127.5.91.131:48711 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Jun 20 20:09:15 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:15 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:15.979249 14649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:15.973548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.131:33511)
0620 20:09:15.973738 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:15.973754 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:15.973829 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:15.974575 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:15.974578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:15.974725 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:15.974805 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:15.974810 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.975539 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.975542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:15.976063 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:15.976070 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.976470 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.976473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:15.976561 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:15.977007 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:15.977018 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:15.979073 (+ 2055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":77}
W20260620 20:09:15.979540 14651 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:15.974095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:33511 (local address 127.5.91.190:33573)
0620 20:09:15.974342 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:15.974346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:15.974360 (+ 14us) server_negotiation.cc:407] Connection header received
0620 20:09:15.974398 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:15.974400 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:15.974455 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:15.974534 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:15.974914 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.975439 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.976164 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:15.976370 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:15.976463 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:15.979254 (+ 2791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.131:33511: BlockingRecv error: recv got EOF from 127.5.91.131:33511 (error 108)
Metrics: {"server-negotiator.queue_time_us":160}
W20260620 20:09:15.979673 14647 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 20 20:09:16 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:16 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:16.528810 14652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:16.522238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.129:53307)
0620 20:09:16.522575 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:16.522591 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:16.522694 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:16.523148 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:16.523150 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:16.523332 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:16.523451 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:16.523456 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:16.524424 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:16.524427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:16.525167 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:16.525177 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:16.525618 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:16.525621 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:16.525693 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:16.526351 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:16.526363 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:16.528663 (+ 2300us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":79,"threads_started":1}
W20260620 20:09:16.528990 14653 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:16.522363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:53307 (local address 127.5.91.190:33573)
0620 20:09:16.522751 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:16.522757 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:16.522780 (+ 23us) server_negotiation.cc:407] Connection header received
0620 20:09:16.522953 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:16.522956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:16.523006 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:16.523081 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:16.523590 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:16.524310 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:16.525280 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:16.525517 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:16.525595 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:16.528869 (+ 3274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.129:53307: BlockingRecv error: recv got EOF from 127.5.91.129:53307 (error 108)
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":109,"threads_started":1}
Jun 20 20:09:16 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:16 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:16.737097 14654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:16.730353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.130:57099)
0620 20:09:16.730751 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:16.730770 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:16.730965 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:16.731456 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:16.731459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:16.731636 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:16.731748 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:16.731754 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:16.732787 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:16.732790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:16.733462 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:16.733473 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:16.734026 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:16.734030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:16.734095 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:16.734681 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:16.734693 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:16.736874 (+ 2181us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":99,"threads_started":1}
W20260620 20:09:16.737478 14653 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:16.730481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:57099 (local address 127.5.91.190:33573)
0620 20:09:16.730781 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:16.730786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:16.730874 (+ 88us) server_negotiation.cc:407] Connection header received
0620 20:09:16.731193 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:16.731196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:16.731269 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:16.731373 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:16.731911 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:16.732621 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:16.733631 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:16.733860 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:16.733979 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:16.737155 (+ 3176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.130:57099: BlockingRecv error: recv got EOF from 127.5.91.130:57099 (error 108)
Metrics: {"server-negotiator.queue_time_us":152}
Jun 20 20:09:16 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:16 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:16.986129 14655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:16.980307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.131:51307)
0620 20:09:16.980624 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:16.980640 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:16.980758 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:16.981002 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:16.981005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:16.981196 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:16.981289 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:16.981295 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:16.982244 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:16.982248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:16.982843 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:16.982851 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:16.983271 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:16.983274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:16.983348 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:16.983842 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:16.983854 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:16.985947 (+ 2093us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":84,"threads_started":1}
W20260620 20:09:16.986268 14653 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:16.980486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:51307 (local address 127.5.91.190:33573)
0620 20:09:16.980637 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:16.980641 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:16.980714 (+ 73us) server_negotiation.cc:407] Connection header received
0620 20:09:16.980854 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:16.980857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:16.980904 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:16.980969 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:16.981544 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:16.982123 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:16.982957 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:16.983173 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:16.983257 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:16.986102 (+ 2845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.131:51307: BlockingRecv error: recv got EOF from 127.5.91.131:51307 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Jun 20 20:09:17 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:17 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:17.537405 14656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:17.530273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.129:49601)
0620 20:09:17.530644 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:17.530659 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:17.530774 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:17.531191 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:17.531194 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:17.531368 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:17.531469 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:17.531476 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:17.532482 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:17.532485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:17.533112 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:17.533121 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:17.533941 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:17.533943 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:17.534039 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:17.534668 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:17.534680 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:17.537240 (+ 2560us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":118,"threads_started":1}
W20260620 20:09:17.537631 14657 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:17.530406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:49601 (local address 127.5.91.190:33573)
0620 20:09:17.530906 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:17.530910 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:17.530927 (+ 17us) server_negotiation.cc:407] Connection header received
0620 20:09:17.530969 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:17.530971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:17.531011 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:17.531098 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:17.531674 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:17.532326 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:17.533288 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:17.533755 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:17.533908 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:17.537491 (+ 3583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.129:49601: BlockingRecv error: recv got EOF from 127.5.91.129:49601 (error 108)
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":99,"threads_started":1}
Jun 20 20:09:17 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:17 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:17.744582 14658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:17.738196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.130:34125)
0620 20:09:17.738456 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:17.738469 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:17.738568 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:17.738995 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:17.738998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:17.739173 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:17.739298 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:17.739304 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:17.740224 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:17.740227 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:17.740912 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:17.740922 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:17.741444 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:17.741449 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:17.741625 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:17.742269 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:17.742284 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:17.744393 (+ 2109us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":84,"threads_started":1}
W20260620 20:09:17.744637 14657 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:17.738293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:34125 (local address 127.5.91.190:33573)
0620 20:09:17.738431 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:17.738434 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:17.738602 (+ 168us) server_negotiation.cc:407] Connection header received
0620 20:09:17.738703 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:17.738705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:17.738786 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:17.738917 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:17.739470 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:17.740102 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:17.741026 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:17.741239 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:17.741317 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:17.744539 (+ 3222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.130:34125: BlockingRecv error: recv got EOF from 127.5.91.130:34125 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 20 20:09:17 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:17 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:17.993074 14659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:17.987245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.131:52733)
0620 20:09:17.987529 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:17.987545 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:17.987652 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:17.987963 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:17.987966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:17.988202 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:17.988307 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:17.988314 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:17.989134 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:17.989137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:17.989719 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:17.989728 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:17.990252 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:17.990256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:17.990327 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:17.990838 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:17.990850 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:17.992916 (+ 2066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":82,"threads_started":1}
W20260620 20:09:17.993319 14657 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:17.987492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:52733 (local address 127.5.91.190:33573)
0620 20:09:17.987606 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:17.987610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:17.987626 (+ 16us) server_negotiation.cc:407] Connection header received
0620 20:09:17.987740 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:17.987743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:17.987800 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:17.987934 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:17.988446 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:17.989004 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:17.989845 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:17.990071 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:17.990192 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:17.993191 (+ 2999us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.131:52733: BlockingRecv error: recv got EOF from 127.5.91.131:52733 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 20 20:09:18 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:18 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:18.544925 14660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:18.538383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.129:54141)
0620 20:09:18.538734 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:18.538747 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:18.538873 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:18.539337 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:18.539340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:18.539564 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:18.539754 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:18.539766 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:18.540926 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:18.540931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:18.541650 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:18.541663 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:18.542246 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:18.542249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:18.542338 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:18.542909 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:18.542924 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:18.544756 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":98,"threads_started":1}
W20260620 20:09:18.545053 14661 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:18.538480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:54141 (local address 127.5.91.190:33573)
0620 20:09:18.538740 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:18.538745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:18.538821 (+ 76us) server_negotiation.cc:407] Connection header received
0620 20:09:18.539141 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:18.539145 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:18.539194 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:18.539284 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:18.539862 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:18.540675 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:18.541814 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:18.542063 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:18.542191 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:18.544937 (+ 2746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.129:54141: BlockingRecv error: recv got EOF from 127.5.91.129:54141 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":68,"threads_started":1}
Jun 20 20:09:18 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:18 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:18.751782 14662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:18.745865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.130:55375)
0620 20:09:18.746305 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:18.746323 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:18.746478 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:18.746958 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:18.746961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:18.747204 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:18.747322 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:18.747330 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:18.748218 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:18.748221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:18.748858 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:18.748868 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:18.749299 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:18.749303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:18.749375 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:18.749876 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:18.749889 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:18.751626 (+ 1737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":121,"threads_started":1}
W20260620 20:09:18.751863 14661 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:18.745890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:55375 (local address 127.5.91.190:33573)
0620 20:09:18.746123 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:18.746128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:18.746434 (+ 306us) server_negotiation.cc:407] Connection header received
0620 20:09:18.746618 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:18.746620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:18.746669 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:18.746748 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:18.747467 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:18.748119 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:18.748982 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:18.749198 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:18.749282 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:18.751764 (+ 2482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.130:55375: BlockingRecv error: recv got EOF from 127.5.91.130:55375 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Jun 20 20:09:18 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:18 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:19.001998 14663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:18.994131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.131:54961)
0620 20:09:18.994723 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:18.994740 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:18.995070 (+ 330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:18.995364 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:18.995367 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:18.995884 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:18.996058 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:18.996070 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:18.997237 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:18.997241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:18.998115 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:18.998127 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:18.998824 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:18.998829 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:18.998931 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:18.999862 (+ 931us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:18.999876 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:19.001845 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":251,"threads_started":1}
W20260620 20:09:19.002866 14661 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:18.994434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:54961 (local address 127.5.91.190:33573)
0620 20:09:18.994616 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:18.994620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:18.995081 (+ 461us) server_negotiation.cc:407] Connection header received
0620 20:09:18.995162 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:18.995164 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:18.995218 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:18.995291 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:18.996401 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:18.997061 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:18.998320 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:18.998550 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:18.998643 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:19.002749 (+ 4106us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.131:54961: BlockingRecv error: recv got EOF from 127.5.91.131:54961 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Jun 20 20:09:19 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:19 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:19.555393 14664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:19.545871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.129:52207)
0620 20:09:19.546131 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:19.546168 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:19.546259 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:19.546570 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:19.546577 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:19.546745 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:19.546843 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:19.546848 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:19.547736 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:19.547740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:19.548419 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:19.548427 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:19.548883 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:19.548886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:19.548952 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:19.553137 (+ 4185us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:19.553150 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:19.555265 (+ 2115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":91,"threads_started":1}
W20260620 20:09:19.555558 14665 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:19.545913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:52207 (local address 127.5.91.190:33573)
0620 20:09:19.546197 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:19.546201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:19.546216 (+ 15us) server_negotiation.cc:407] Connection header received
0620 20:09:19.546404 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:19.546407 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:19.546457 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:19.546551 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:19.546963 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:19.547601 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:19.548513 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:19.548738 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:19.548823 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:19.555418 (+ 6595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.129:52207: BlockingRecv error: recv got EOF from 127.5.91.129:52207 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":79,"threads_started":1}
Jun 20 20:09:19 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:19 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:19.759634 14666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:19.752793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.130:52791)
0620 20:09:19.753133 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:19.753148 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:19.753246 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:19.753609 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:19.753614 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:19.753850 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:19.754094 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:19.754102 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:19.754989 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:19.754992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:19.755557 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:19.755565 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:19.756258 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:19.756261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:19.756334 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:19.757391 (+ 1057us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:19.757406 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:19.759513 (+ 2107us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":97,"threads_started":1}
W20260620 20:09:19.759799 14665 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:19.753060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:52791 (local address 127.5.91.190:33573)
0620 20:09:19.753200 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:19.753204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:19.753219 (+ 15us) server_negotiation.cc:407] Connection header received
0620 20:09:19.753323 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:19.753326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:19.753368 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:19.753443 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:19.754283 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:19.754835 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:19.755756 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:19.756058 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:19.756201 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:19.759690 (+ 3489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.130:52791: BlockingRecv error: recv got EOF from 127.5.91.130:52791 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 20 20:09:20 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:20 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:20.009850 14667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:20.002941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.131:43631)
0620 20:09:20.003240 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:20.003255 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:20.003371 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:20.003622 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:20.003625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:20.003811 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:20.003914 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:20.003919 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:20.004931 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:20.004935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:20.005507 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:20.005516 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:20.005936 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:20.005940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:20.006013 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:20.006537 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:20.006549 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:20.009677 (+ 3128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":95,"threads_started":1}
W20260620 20:09:20.009969 14665 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:20.003070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:43631 (local address 127.5.91.190:33573)
0620 20:09:20.003241 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:20.003246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:20.003326 (+ 80us) server_negotiation.cc:407] Connection header received
0620 20:09:20.003468 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:20.003471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:20.003520 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:20.003586 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:20.004018 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:20.004822 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:20.005618 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:20.005836 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:20.005922 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:20.009863 (+ 3941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.131:43631: BlockingRecv error: recv got EOF from 127.5.91.131:43631 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Jun 20 20:09:20 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:20 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:20.562947 14668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:20.556545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.129:35957)
0620 20:09:20.556799 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:20.556812 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:20.556893 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:20.557140 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:20.557142 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:20.557318 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:20.557461 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:20.557467 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:20.558254 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:20.558256 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:20.558860 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:20.558868 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:20.559612 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:20.559615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:20.559684 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:20.560233 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:20.560250 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:20.562730 (+ 2480us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":77,"threads_started":1}
W20260620 20:09:20.563016 14669 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:20.556681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:35957 (local address 127.5.91.190:33573)
0620 20:09:20.556943 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:20.556946 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:20.556959 (+ 13us) server_negotiation.cc:407] Connection header received
0620 20:09:20.556997 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:20.556999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:20.557037 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:20.557120 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:20.557597 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:20.558165 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:20.559072 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:20.559375 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:20.559583 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:20.562903 (+ 3320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.129:35957: BlockingRecv error: recv got EOF from 127.5.91.129:35957 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":95,"threads_started":1}
Jun 20 20:09:20 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:20 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:20.767421 14670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:20.760783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.130:48449)
0620 20:09:20.761254 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:20.761278 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:20.761452 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:20.761945 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:20.761949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:20.762423 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:20.762559 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:20.762568 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:20.763603 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:20.763607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:20.764208 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:20.764217 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:20.764705 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:20.764709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:20.764784 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:20.765319 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:20.765330 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:20.767264 (+ 1934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":177,"threads_started":1}
W20260620 20:09:20.767557 14669 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:20.761268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:48449 (local address 127.5.91.190:33573)
0620 20:09:20.761497 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:20.761502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:20.761595 (+ 93us) server_negotiation.cc:407] Connection header received
0620 20:09:20.761649 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:20.761652 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:20.761713 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:20.761912 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:20.762726 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:20.763444 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:20.764337 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:20.764591 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:20.764685 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:20.767439 (+ 2754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.130:48449: BlockingRecv error: recv got EOF from 127.5.91.130:48449 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Jun 20 20:09:21 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:21 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:21.019222 14671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:21.010850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.131:49373)
0620 20:09:21.011143 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:21.011156 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:21.011269 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:21.011738 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:21.011740 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:21.011939 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:21.012044 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:21.012050 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:21.012969 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:21.012975 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:21.014219 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:21.014241 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:21.015308 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:21.015312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:21.015408 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:21.016266 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:21.016281 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:21.019046 (+ 2765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":78,"threads_started":1}
W20260620 20:09:21.019344 14669 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:21.010973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:49373 (local address 127.5.91.190:33573)
0620 20:09:21.011118 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:21.011121 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:21.011381 (+ 260us) server_negotiation.cc:407] Connection header received
0620 20:09:21.011472 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:21.011474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:21.011555 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:21.011661 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:21.012199 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:21.012757 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:21.014517 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:21.015034 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:21.015143 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:21.019223 (+ 4080us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.131:49373: BlockingRecv error: recv got EOF from 127.5.91.131:49373 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Jun 20 20:09:21 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:21 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:21.571305 14672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:21.563824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.129:33149)
0620 20:09:21.564327 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:21.564342 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:21.564449 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:21.564742 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:21.564745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:21.564926 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:21.565034 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:21.565040 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:21.565917 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:21.565923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:21.566694 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:21.566711 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:21.567271 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:21.567276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:21.567383 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:21.568275 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:21.568293 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:21.571141 (+ 2848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":232,"threads_started":1}
W20260620 20:09:21.571372 14673 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:21.564019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:33149 (local address 127.5.91.190:33573)
0620 20:09:21.564257 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:21.564261 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:21.564490 (+ 229us) server_negotiation.cc:407] Connection header received
0620 20:09:21.564542 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:21.564544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:21.564591 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:21.564677 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:21.565167 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:21.565711 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:21.566884 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:21.567115 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:21.567206 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:21.571262 (+ 4056us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.129:33149: BlockingRecv error: recv got EOF from 127.5.91.129:33149 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":65,"threads_started":1}
Jun 20 20:09:21 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:21 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:21.775997 14674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:21.768505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.130:53667)
0620 20:09:21.768787 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:21.768801 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:21.768952 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:21.769341 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:21.769344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:21.769518 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:21.769620 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:21.769625 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:21.771053 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:21.771058 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:21.771855 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:21.771869 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:21.772633 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:21.772638 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:21.772893 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:21.773474 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:21.773486 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:21.775831 (+ 2345us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":96,"threads_started":1}
W20260620 20:09:21.776185 14673 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:21.768586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:53667 (local address 127.5.91.190:33573)
0620 20:09:21.768724 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:21.768727 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:21.768922 (+ 195us) server_negotiation.cc:407] Connection header received
0620 20:09:21.769123 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:21.769126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:21.769177 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:21.769271 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:21.769847 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:21.770860 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:21.772098 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:21.772418 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:21.772520 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:21.776054 (+ 3534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.130:53667: BlockingRecv error: recv got EOF from 127.5.91.130:53667 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 20 20:09:22 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:22 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:22.026582 14675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:22.020287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.131:42631)
0620 20:09:22.020713 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:22.020728 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:22.020855 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:22.021095 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:22.021098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:22.021300 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:22.021388 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:22.021394 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:22.022274 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:22.022277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:22.023019 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:22.023028 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:22.023446 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:22.023449 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:22.023525 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:22.024136 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:22.024153 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:22.026414 (+ 2261us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":101,"threads_started":1}
W20260620 20:09:22.026696 14673 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:22.020496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:42631 (local address 127.5.91.190:33573)
0620 20:09:22.020715 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:22.020719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:22.020805 (+ 86us) server_negotiation.cc:407] Connection header received
0620 20:09:22.020948 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:22.020951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:22.020999 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:22.021065 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:22.021509 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:22.022175 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:22.023120 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:22.023340 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:22.023426 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:22.026593 (+ 3167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.131:42631: BlockingRecv error: recv got EOF from 127.5.91.131:42631 (error 108)
Metrics: {"server-negotiator.queue_time_us":120}
Jun 20 20:09:22 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:22 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:22.578562 14676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:22.572306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.129:54623)
0620 20:09:22.572622 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:22.572635 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:22.572724 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:22.573407 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:22.573409 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:22.573606 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:22.573708 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:22.573714 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:22.574635 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:22.574640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:22.575400 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:22.575411 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:22.575905 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:22.575908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:22.575986 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:22.576496 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:22.576508 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:22.578442 (+ 1934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":81,"threads_started":1}
W20260620 20:09:22.578702 14677 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:22.572411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:54623 (local address 127.5.91.190:33573)
0620 20:09:22.572929 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:22.572933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:22.572954 (+ 21us) server_negotiation.cc:407] Connection header received
0620 20:09:22.573008 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:22.573011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:22.573219 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:22.573311 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:22.573858 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:22.574434 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:22.575537 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:22.575768 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:22.575850 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:22.578598 (+ 2748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.129:54623: BlockingRecv error: recv got EOF from 127.5.91.129:54623 (error 108)
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":256,"threads_started":1}
Jun 20 20:09:22 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:22 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:22.784016 14678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:22.777068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.130:60629)
0620 20:09:22.777383 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:22.777405 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:22.777508 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:22.777746 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:22.777748 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:22.777926 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:22.778032 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:22.778038 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:22.778931 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:22.778937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:22.780013 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:22.780035 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:22.780828 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:22.780832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:22.780947 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:22.781618 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:22.781632 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:22.783877 (+ 2245us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":90,"threads_started":1}
W20260620 20:09:22.784354 14677 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:22.777184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:60629 (local address 127.5.91.190:33573)
0620 20:09:22.777343 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:22.777346 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:22.777508 (+ 162us) server_negotiation.cc:407] Connection header received
0620 20:09:22.777587 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:22.777590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:22.777641 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:22.777717 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:22.778122 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:22.778687 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:22.780196 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:22.780668 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:22.780801 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:22.784195 (+ 3394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.130:60629: BlockingRecv error: recv got EOF from 127.5.91.130:60629 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Jun 20 20:09:23 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:23 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:23.033474 14679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:23.027922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.131:52875)
0620 20:09:23.028318 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:23.028331 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:23.028436 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:23.028697 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:23.028699 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:23.028912 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:23.029000 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:23.029005 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:23.029742 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:23.029745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:23.030330 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:23.030338 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:23.030740 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:23.030744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:23.030817 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:23.031309 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:23.031321 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:23.033320 (+ 1999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":86,"threads_started":1}
W20260620 20:09:23.033639 14677 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:23.028047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:52875 (local address 127.5.91.190:33573)
0620 20:09:23.028162 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:23.028166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:23.028406 (+ 240us) server_negotiation.cc:407] Connection header received
0620 20:09:23.028512 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:23.028516 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:23.028563 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:23.028630 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:23.029127 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:23.029642 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:23.030436 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:23.030645 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:23.030726 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:23.033528 (+ 2802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.131:52875: BlockingRecv error: recv got EOF from 127.5.91.131:52875 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Jun 20 20:09:23 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:23 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:23.584806 14680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:23.579902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.129:51381)
0620 20:09:23.580134 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:23.580146 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:23.580229 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:23.580465 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:23.580467 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:23.580640 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:23.580730 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:23.580735 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:23.581476 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:23.581479 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:23.582036 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:23.582043 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:23.582493 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:23.582495 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:23.582553 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:23.583063 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:23.583073 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:23.584690 (+ 1617us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":80,"threads_started":1}
W20260620 20:09:23.585045 14681 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:23.580037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:51381 (local address 127.5.91.190:33573)
0620 20:09:23.580243 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:23.580247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:23.580260 (+ 13us) server_negotiation.cc:407] Connection header received
0620 20:09:23.580298 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:23.580300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:23.580340 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:23.580413 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:23.580831 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:23.581356 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:23.582170 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:23.582402 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:23.582488 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:23.584916 (+ 2428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.129:51381: BlockingRecv error: recv got EOF from 127.5.91.129:51381 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":64,"threads_started":1}
Jun 20 20:09:23 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:23 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:23.790989 14682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:23.784962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.130:40153)
0620 20:09:23.785223 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:23.785236 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:23.785340 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:23.785797 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:23.785800 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:23.785977 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:23.786070 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:23.786075 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:23.787233 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:23.787236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:23.787952 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:23.787962 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:23.788498 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:23.788505 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:23.788576 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:23.789102 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:23.789114 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:23.790864 (+ 1750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":79,"threads_started":1}
W20260620 20:09:23.791191 14681 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:23.785083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:40153 (local address 127.5.91.190:33573)
0620 20:09:23.785240 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:23.785243 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:23.785314 (+ 71us) server_negotiation.cc:407] Connection header received
0620 20:09:23.785422 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:23.785424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:23.785470 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:23.785550 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:23.786210 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:23.787104 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:23.788087 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:23.788339 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:23.788422 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:23.791078 (+ 2656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.130:40153: BlockingRecv error: recv got EOF from 127.5.91.130:40153 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 20 20:09:24 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:24 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:24.041260 14683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:24.034572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.131:41717)
0620 20:09:24.034950 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:24.034970 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:24.035137 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:24.035484 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:24.035487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:24.035670 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:24.035768 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:24.035774 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:24.036643 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:24.036647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:24.037342 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:24.037354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:24.037893 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:24.037896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:24.037980 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:24.038789 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:24.038803 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:24.041128 (+ 2325us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":89,"threads_started":1}
W20260620 20:09:24.041390 14681 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:24.034810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:41717 (local address 127.5.91.190:33573)
0620 20:09:24.034962 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:24.034967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:24.035095 (+ 128us) server_negotiation.cc:407] Connection header received
0620 20:09:24.035263 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:24.035266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:24.035320 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:24.035395 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:24.035896 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:24.036467 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:24.037509 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:24.037743 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:24.037830 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:24.041273 (+ 3443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.131:41717: BlockingRecv error: recv got EOF from 127.5.91.131:41717 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 20 20:09:24 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:24 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:24.597272 14684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:24.585661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.129:54635)
0620 20:09:24.585999 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:24.586011 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:24.586104 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:24.586704 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:24.586706 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:24.586885 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:24.586992 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:24.586997 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:24.588291 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:24.588294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:24.589076 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:24.589100 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:24.589999 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:24.590003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:24.590080 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:24.594293 (+ 4213us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:24.594313 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:24.597099 (+ 2786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":81,"threads_started":1}
W20260620 20:09:24.597798 14685 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:24.585918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:54635 (local address 127.5.91.190:33573)
0620 20:09:24.586345 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:24.586348 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:24.586371 (+ 23us) server_negotiation.cc:407] Connection header received
0620 20:09:24.586430 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:24.586433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:24.586489 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:24.586623 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:24.587161 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:24.588138 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:24.589562 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:24.589807 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:24.589932 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:24.597386 (+ 7454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.129:54635: BlockingRecv error: recv got EOF from 127.5.91.129:54635 (error 108)
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":199,"threads_started":1}
Jun 20 20:09:24 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:24 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:24.798003 14686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:24.791975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.130:48879)
0620 20:09:24.792489 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:24.792506 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:24.792675 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:24.792974 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:24.792977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:24.793394 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:24.793551 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:24.793561 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:24.794444 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:24.794447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:24.795023 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:24.795032 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:24.795452 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:24.795455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:24.795526 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:24.796058 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:24.796070 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:24.797823 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":154,"threads_started":1}
W20260620 20:09:24.798170 14685 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:24.792204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:48879 (local address 127.5.91.190:33573)
0620 20:09:24.792467 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:24.792473 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:24.792629 (+ 156us) server_negotiation.cc:407] Connection header received
0620 20:09:24.792792 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:24.792795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:24.792861 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:24.792936 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:24.793681 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:24.794335 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:24.795132 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:24.795354 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:24.795438 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:24.798035 (+ 2597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.130:48879: BlockingRecv error: recv got EOF from 127.5.91.130:48879 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Jun 20 20:09:25 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:25 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:25.047991 14687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:25.042007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.131:35085)
0620 20:09:25.042347 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:25.042363 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:25.042480 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:25.042743 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:25.042745 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:25.042989 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:25.043090 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:25.043096 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:25.043827 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:25.043830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:25.044396 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:25.044404 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:25.044820 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:25.044824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:25.044898 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:25.045525 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:25.045537 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:25.047764 (+ 2227us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":124,"threads_started":1}
W20260620 20:09:25.048024 14685 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:25.042112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:35085 (local address 127.5.91.190:33573)
0620 20:09:25.042348 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:25.042353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:25.042433 (+ 80us) server_negotiation.cc:407] Connection header received
0620 20:09:25.042585 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:25.042588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:25.042643 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:25.042708 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:25.043201 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:25.043728 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:25.044506 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:25.044724 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:25.044805 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:25.047925 (+ 3120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.131:35085: BlockingRecv error: recv got EOF from 127.5.91.131:35085 (error 108)
Metrics: {"server-negotiator.queue_time_us":161}
Jun 20 20:09:25 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:25 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:25.605041 14688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:25.598386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.129:51251)
0620 20:09:25.598632 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:25.598647 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:25.598746 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:25.599034 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:25.599037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:25.599241 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:25.599335 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:25.599341 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:25.600323 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:25.600329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:25.601255 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:25.601274 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:25.601988 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:25.601992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:25.602074 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:25.602765 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:25.602785 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:25.604889 (+ 2104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":74,"threads_started":1}
W20260620 20:09:25.605171 14689 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:25.598472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:51251 (local address 127.5.91.190:33573)
0620 20:09:25.598699 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:25.598703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:25.598718 (+ 15us) server_negotiation.cc:407] Connection header received
0620 20:09:25.598830 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:25.598832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:25.598883 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:25.598977 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:25.599462 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:25.600084 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:25.601576 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:25.601817 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:25.601909 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:25.605053 (+ 3144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.129:51251: BlockingRecv error: recv got EOF from 127.5.91.129:51251 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":91,"threads_started":1}
Jun 20 20:09:25 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:25 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:25.805284 14690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:25.799117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.130:38929)
0620 20:09:25.799455 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:25.799470 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:25.799727 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:25.800047 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:25.800050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:25.800234 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:25.800334 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:25.800340 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:25.801139 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:25.801143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:25.801726 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:25.801735 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:25.802211 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:25.802214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:25.802287 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:25.802973 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:25.802998 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:25.805164 (+ 2166us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":106,"threads_started":1}
W20260620 20:09:25.805434 14689 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:25.799292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:38929 (local address 127.5.91.190:33573)
0620 20:09:25.799498 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:25.799502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:25.799691 (+ 189us) server_negotiation.cc:407] Connection header received
0620 20:09:25.799859 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:25.799863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:25.799926 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:25.800000 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:25.800505 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:25.801038 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:25.801829 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:25.802072 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:25.802200 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:25.805319 (+ 3119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.130:38929: BlockingRecv error: recv got EOF from 127.5.91.130:38929 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Jun 20 20:09:26 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:26 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:26.055109 14691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:26.048970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.131:35607)
0620 20:09:26.049211 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:26.049223 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:26.049300 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:26.049558 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:26.049560 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:26.049728 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:26.049819 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:26.049823 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:26.050812 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:26.050816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:26.051588 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:26.051597 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:26.052218 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:26.052221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:26.052287 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:26.052756 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:26.052768 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:26.054993 (+ 2225us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":71,"threads_started":1}
W20260620 20:09:26.055320 14689 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:26.049197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:35607 (local address 127.5.91.190:33573)
0620 20:09:26.049318 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:26.049321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:26.049334 (+ 13us) server_negotiation.cc:407] Connection header received
0620 20:09:26.049375 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:26.049377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:26.049421 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:26.049515 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:26.049947 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:26.050630 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:26.051717 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:26.052062 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:26.052159 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:26.055202 (+ 3043us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.131:35607: BlockingRecv error: recv got EOF from 127.5.91.131:35607 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Jun 20 20:09:26 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:26 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:26.613303 14692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:26.606237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.129:56987)
0620 20:09:26.606611 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:26.606631 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:26.606788 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:26.607112 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:26.607115 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:26.607293 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:26.607419 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:26.607425 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:26.608482 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:26.608485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:26.609081 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:26.609089 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:26.609636 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:26.609639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:26.609706 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:26.610459 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:26.610471 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:26.613109 (+ 2638us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":83,"threads_started":1}
W20260620 20:09:26.613368 14693 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:26.606416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:56987 (local address 127.5.91.190:33573)
0620 20:09:26.606782 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:26.606786 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:26.606804 (+ 18us) server_negotiation.cc:407] Connection header received
0620 20:09:26.606894 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:26.606897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:26.606947 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:26.607033 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:26.607595 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:26.608357 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:26.609209 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:26.609463 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:26.609561 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:26.613250 (+ 3689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.129:56987: BlockingRecv error: recv got EOF from 127.5.91.129:56987 (error 108)
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":119,"threads_started":1}
Jun 20 20:09:26 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:26 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:26.812188 14694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:26.806357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.130:42887)
0620 20:09:26.806617 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:26.806633 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:26.806748 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:26.807110 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:26.807113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:26.807293 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:26.807406 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:26.807412 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:26.808229 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:26.808232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:26.808833 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:26.808842 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:26.809311 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:26.809314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:26.809382 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:26.810224 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:26.810236 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:26.812036 (+ 1800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":88,"threads_started":1}
W20260620 20:09:26.812299 14693 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:26.806413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:42887 (local address 127.5.91.190:33573)
0620 20:09:26.806546 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:26.806550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:26.806777 (+ 227us) server_negotiation.cc:407] Connection header received
0620 20:09:26.806854 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:26.806857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:26.806935 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:26.807042 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:26.807547 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:26.808110 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:26.808969 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:26.809189 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:26.809275 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:26.812199 (+ 2924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.130:42887: BlockingRecv error: recv got EOF from 127.5.91.130:42887 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 20 20:09:27 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:27 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:27.062520 14695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:27.055851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.131:33377)
0620 20:09:27.056152 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:27.056165 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:27.056246 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:27.056708 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:27.056712 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:27.056944 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:27.057078 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:27.057085 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:27.057986 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:27.057989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:27.058725 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:27.058735 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:27.059294 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:27.059298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:27.059403 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:27.060076 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:27.060089 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:27.062334 (+ 2245us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":67,"threads_started":1}
W20260620 20:09:27.062997 14693 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:27.056095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:33377 (local address 127.5.91.190:33573)
0620 20:09:27.056253 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:27.056257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:27.056277 (+ 20us) server_negotiation.cc:407] Connection header received
0620 20:09:27.056334 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:27.056336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:27.056396 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:27.056499 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:27.057279 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:27.057844 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:27.058867 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:27.059098 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:27.059190 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:27.062578 (+ 3388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.131:33377: BlockingRecv error: recv got EOF from 127.5.91.131:33377 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 20 20:09:27 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:27 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:27.620985 14696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:27.614415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.129:43343)
0620 20:09:27.614669 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:27.614684 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:27.614775 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:27.615309 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:27.615313 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:27.615534 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:27.615720 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:27.615727 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:27.616753 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:27.616756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:27.617322 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:27.617330 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:27.618055 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:27.618058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:27.618123 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:27.618772 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:27.618786 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:27.620859 (+ 2073us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":72,"threads_started":1}
W20260620 20:09:27.621235 14697 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:27.614645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:43343 (local address 127.5.91.190:33573)
0620 20:09:27.614952 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:27.614955 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:27.614972 (+ 17us) server_negotiation.cc:407] Connection header received
0620 20:09:27.615025 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:27.615027 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:27.615088 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:27.615205 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:27.615889 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:27.616472 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:27.617461 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:27.617903 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:27.618022 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:27.621074 (+ 3052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.129:43343: BlockingRecv error: recv got EOF from 127.5.91.129:43343 (error 108)
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":109,"threads_started":1}
Jun 20 20:09:27 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:27 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:27.819887 14698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:27.813369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.130:59965)
0620 20:09:27.813745 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:27.813763 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:27.813898 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:27.814203 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:27.814206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:27.814419 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:27.814542 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:27.814548 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:27.815291 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:27.815295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:27.815840 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:27.815847 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:27.816290 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:27.816293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:27.816369 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:27.816861 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:27.816872 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:27.819696 (+ 2824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":121,"threads_started":1}
W20260620 20:09:27.820171 14697 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:27.813526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:59965 (local address 127.5.91.190:33573)
0620 20:09:27.813743 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:27.813748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:27.813853 (+ 105us) server_negotiation.cc:407] Connection header received
0620 20:09:27.814017 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:27.814020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:27.814065 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:27.814162 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:27.814660 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:27.815189 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:27.815962 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:27.816173 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:27.816256 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:27.819882 (+ 3626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.130:59965: BlockingRecv error: recv got EOF from 127.5.91.130:59965 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Jun 20 20:09:28 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:28 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:28.069360 14699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:28.063606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.131:44137)
0620 20:09:28.063863 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:28.063876 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:28.064000 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:28.064312 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:28.064315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:28.064488 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:28.064587 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:28.064592 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:28.065414 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:28.065417 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:28.066040 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:28.066048 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:28.066565 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:28.066568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:28.066634 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:28.067151 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:28.067165 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:28.069239 (+ 2074us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":88,"threads_started":1}
W20260620 20:09:28.069453 14697 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:28.063723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:44137 (local address 127.5.91.190:33573)
0620 20:09:28.063873 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:28.063876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:28.063979 (+ 103us) server_negotiation.cc:407] Connection header received
0620 20:09:28.064124 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:28.064127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:28.064176 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:28.064255 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:28.064722 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:28.065301 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:28.066183 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:28.066411 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:28.066494 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:28.069345 (+ 2851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.131:44137: BlockingRecv error: recv got EOF from 127.5.91.131:44137 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Jun 20 20:09:28 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:28 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:28.628929 14700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:28.622061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.129:56759)
0620 20:09:28.622353 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:28.622370 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:28.622480 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:28.622802 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:28.622805 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:28.623100 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:28.623220 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:28.623228 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:28.624223 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:28.624226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:28.624789 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:28.624797 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:28.625556 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:28.625559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:28.625624 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:28.626257 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:28.626272 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:28.628724 (+ 2452us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":98,"threads_started":1}
W20260620 20:09:28.629074 14701 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:28.622239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:56759 (local address 127.5.91.190:33573)
0620 20:09:28.622520 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:28.622524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:28.622538 (+ 14us) server_negotiation.cc:407] Connection header received
0620 20:09:28.622587 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:28.622590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:28.622638 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:28.622747 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:28.623350 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:28.624085 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:28.624914 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:28.625370 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:28.625537 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:28.628925 (+ 3388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.129:56759: BlockingRecv error: recv got EOF from 127.5.91.129:56759 (error 108)
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":73,"threads_started":1}
Jun 20 20:09:28 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:28 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:28.826092 14702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:28.820806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.130:46903)
0620 20:09:28.821123 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:28.821140 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:28.821260 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:28.821489 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:28.821493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:28.821740 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:28.821832 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:28.821838 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:28.822599 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:28.822602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:28.823156 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:28.823165 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:28.823571 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:28.823575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:28.823646 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:28.824190 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:28.824201 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:28.825934 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":102,"threads_started":1}
W20260620 20:09:28.826184 14701 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:28.820995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:46903 (local address 127.5.91.190:33573)
0620 20:09:28.821162 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:28.821167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:28.821202 (+ 35us) server_negotiation.cc:407] Connection header received
0620 20:09:28.821345 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:28.821348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:28.821394 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:28.821459 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:28.821949 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:28.822504 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:28.823266 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:28.823477 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:28.823559 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:28.826062 (+ 2503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.130:46903: BlockingRecv error: recv got EOF from 127.5.91.130:46903 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Jun 20 20:09:29 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:29 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:29.076275 14703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:29.070112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.131:60781)
0620 20:09:29.070384 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:29.070396 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:29.070502 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:29.070796 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:29.070798 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:29.071023 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:29.071119 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:29.071124 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:29.072002 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:29.072006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:29.072598 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:29.072607 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:29.073117 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:29.073120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:29.073198 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:29.073718 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:29.073737 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:29.076144 (+ 2407us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":77,"threads_started":1}
W20260620 20:09:29.076400 14701 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:29.070192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:60781 (local address 127.5.91.190:33573)
0620 20:09:29.070321 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:29.070325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:29.070462 (+ 137us) server_negotiation.cc:407] Connection header received
0620 20:09:29.070595 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:29.070598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:29.070647 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:29.070721 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:29.071215 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:29.071827 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:29.072726 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:29.072963 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:29.073063 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:29.076297 (+ 3234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.131:60781: BlockingRecv error: recv got EOF from 127.5.91.131:60781 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 20 20:09:29 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:29 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:29.641058 14704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:29.629764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.129:49567)
0620 20:09:29.630059 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:29.630072 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:29.630181 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:29.630555 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:29.630558 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:29.630732 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:29.630846 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:29.630852 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:29.631643 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:29.631646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:29.632224 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:29.632232 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:29.632699 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:29.632702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:29.632766 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:29.638585 (+ 5819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:29.638603 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:29.640921 (+ 2318us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":83,"threads_started":1}
W20260620 20:09:29.641218 14705 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:29.629865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:49567 (local address 127.5.91.190:33573)
0620 20:09:29.630327 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:29.630331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:29.630343 (+ 12us) server_negotiation.cc:407] Connection header received
0620 20:09:29.630377 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:29.630379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:29.630419 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:29.630498 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:29.630976 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:29.631514 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:29.632341 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:29.632592 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:29.632675 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:29.641064 (+ 8389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.129:49567: BlockingRecv error: recv got EOF from 127.5.91.129:49567 (error 108)
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":140,"threads_started":1}
Jun 20 20:09:29 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:29 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
W20260620 20:09:29.833524 14705 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:29.827120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:51331 (local address 127.5.91.190:33573)
0620 20:09:29.827280 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:29.827284 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:29.827432 (+ 148us) server_negotiation.cc:407] Connection header received
0620 20:09:29.827531 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:29.827534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:29.827585 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:29.827648 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:29.828287 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:29.829136 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:29.830124 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:29.830478 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:29.830583 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:29.833414 (+ 2831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.130:51331: BlockingRecv error: recv got EOF from 127.5.91.130:51331 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 20:09:29.833608 14706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:29.827005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.130:51331)
0620 20:09:29.827294 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:29.827336 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:29.827442 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:29.827714 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:29.827717 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:29.827902 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:29.828009 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:29.828015 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:29.829285 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:29.829288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:29.830011 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:29.830021 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:29.830611 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:29.830615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:29.830680 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:29.831178 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:29.831190 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:29.833288 (+ 2098us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":96,"threads_started":1}
Jun 20 20:09:30 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:30 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.131@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:30.082804 14707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:30.077179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.131:54169)
0620 20:09:30.077449 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:30.077462 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:30.077563 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:30.077834 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:30.077837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:30.078020 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:30.078110 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:30.078115 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:30.079013 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:30.079016 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:30.079762 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:30.079771 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:30.080268 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:30.080272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:30.080355 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:30.080891 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:30.080905 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:30.082659 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":78,"threads_started":1}
W20260620 20:09:30.082885 14705 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:30.077309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:54169 (local address 127.5.91.190:33573)
0620 20:09:30.077452 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:30.077455 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:30.077552 (+ 97us) server_negotiation.cc:407] Connection header received
0620 20:09:30.077639 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:30.077641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:30.077688 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:30.077764 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:30.078283 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:30.078844 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:30.079886 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:30.080159 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:30.080255 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:30.082793 (+ 2538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.131:54169: BlockingRecv error: recv got EOF from 127.5.91.131:54169 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Jun 20 20:09:30 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:30 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.129@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:30.648518 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:30.642282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.129:33919)
0620 20:09:30.642562 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:30.642577 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:30.642682 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:30.642970 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:30.642973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:30.643199 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:30.643326 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:30.643334 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:30.644508 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:30.644511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:30.645100 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:30.645108 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:30.645681 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:30.645686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:30.645791 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:30.646494 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:30.646507 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:30.648390 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":102,"threads_started":1}
W20260620 20:09:30.648691 14709 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:30.642375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:33919 (local address 127.5.91.190:33573)
0620 20:09:30.642623 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:30.642626 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:30.642642 (+ 16us) server_negotiation.cc:407] Connection header received
0620 20:09:30.642798 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:30.642801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:30.642850 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:30.642932 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:30.643538 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:30.644395 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:30.645226 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:30.645456 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:30.645550 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:30.648555 (+ 3005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.129:33919: BlockingRecv error: recv got EOF from 127.5.91.129:33919 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":103,"threads_started":1}
Jun 20 20:09:30 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 20:09:30 dist-test-slave-l8h4 krb5kdc[13651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.91.130@KRBTEST.COM for oryx/127.5.91.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 20:09:30.840078 14710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:30.834464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33573 (local address 127.5.91.130:42845)
0620 20:09:30.834724 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:30.834737 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:30.834827 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:30.835088 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:30.835090 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:30.835266 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:30.835358 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:30.835363 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:30.836164 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:30.836168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:30.836762 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:30.836771 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:30.837263 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:30.837267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:30.837335 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:30.837979 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:30.837992 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:30.839912 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.91.190:33573: Server oryx/127.5.91.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":82,"threads_started":1}
W20260620 20:09:30.840167 14709 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:30.834594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:42845 (local address 127.5.91.190:33573)
0620 20:09:30.834746 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:30.834750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:30.834832 (+ 82us) server_negotiation.cc:407] Connection header received
0620 20:09:30.834914 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:30.834916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:30.834960 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:30.835028 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:30.835524 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:30.836060 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:30.836884 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:30.837105 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:30.837192 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:30.840057 (+ 2865us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.91.130:42845: BlockingRecv error: recv got EOF from 127.5.91.130:42845 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 20:09:30.958317 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 14234
I20260620 20:09:30.963748 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 14386
I20260620 20:09:30.968252 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 14518
I20260620 20:09:30.972687 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 14164
2026-06-20T20:09:30Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18264 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-20T20:09:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:09:30Z Disabled control of system clock
I20260620 20:09:31.003712 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:41591
--webserver_interface=127.5.91.190
--webserver_port=0
--builtin_ntp_servers=127.5.91.148:43219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:41591
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260620 20:09:31.128511 14714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:31.128796 14714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:31.128855 14714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:31.133296 14714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:09:31.133388 14714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:31.133415 14714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:31.133436 14714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:09:31.133457 14714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:09:31.138815 14714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:43219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:41591
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:41591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:31.139907 14714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:31.141283 14714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:31.149755 14719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:31.149799 14722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:31.149787 14720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:31.149848 14714 server_base.cc:1034] running on GCE node
I20260620 20:09:31.150465 14714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:31.151517 14714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:31.152724 14714 hybrid_clock.cc:648] HybridClock initialized: now 1781986171152692 us; error 40 us; skew 500 ppm
I20260620 20:09:31.155248 14714 webserver.cc:460] Webserver started at http://127.5.91.190:44049/ using document root <none> and password file <none>
I20260620 20:09:31.155820 14714 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:31.155890 14714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:31.156260 14714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:31.159368 14714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "3aa6eb33b35b41d9ad61af761a8d8c69"
format_stamp: "Formatted at 2026-06-20 20:09:31 on dist-test-slave-l8h4"
server_key: "e6008dbf899e98d79c2d7185e1fb8472"
server_key_iv: "f8e06c697bce03baf0a36116d8856bc5"
server_key_version: "encryptionkey@0"
I20260620 20:09:31.160257 14714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "3aa6eb33b35b41d9ad61af761a8d8c69"
format_stamp: "Formatted at 2026-06-20 20:09:31 on dist-test-slave-l8h4"
server_key: "e6008dbf899e98d79c2d7185e1fb8472"
server_key_iv: "f8e06c697bce03baf0a36116d8856bc5"
server_key_version: "encryptionkey@0"
I20260620 20:09:31.165457 14714 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260620 20:09:31.168077 14728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:31.169572 14714 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260620 20:09:31.169694 14714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "3aa6eb33b35b41d9ad61af761a8d8c69"
format_stamp: "Formatted at 2026-06-20 20:09:31 on dist-test-slave-l8h4"
server_key: "e6008dbf899e98d79c2d7185e1fb8472"
server_key_iv: "f8e06c697bce03baf0a36116d8856bc5"
server_key_version: "encryptionkey@0"
I20260620 20:09:31.169835 14714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:31.200651 14714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:31.201396 14714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:31.201568 14714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:31.208704 14714 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:41591
I20260620 20:09:31.208729 14780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:41591 every 8 connection(s)
I20260620 20:09:31.209699 14714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:09:31.211189 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 14714
I20260620 20:09:31.211306 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:09:31.211623 5486 external_mini_cluster.cc:1428] Setting key cc2aa795a3b4b2fdb6075bafcbd1ae58
I20260620 20:09:31.213063 14781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:31.219936 14781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3aa6eb33b35b41d9ad61af761a8d8c69: Bootstrap starting.
I20260620 20:09:31.223080 14784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:31.213010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42416 (local address 127.5.91.190:41591)
0620 20:09:31.213655 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:31.213674 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:31.213715 (+ 41us) server_negotiation.cc:407] Connection header received
0620 20:09:31.214592 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:31.214613 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:31.215116 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:31.215496 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:31.215969 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:31.217419 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:31.218573 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:31.219257 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:31.219403 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:31.220604 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:31.220610 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:31.220643 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:09:31.221267 (+ 624us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:31.221340 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:09:31.221369 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:31.221581 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:31.221737 (+ 156us) server_negotiation.cc:299] Negotiation successful
0620 20:09:31.222092 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":111,"threads_started":1}
I20260620 20:09:31.223088 14781 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3aa6eb33b35b41d9ad61af761a8d8c69: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:31.223935 14781 log.cc:824] T 00000000000000000000000000000000 P 3aa6eb33b35b41d9ad61af761a8d8c69: Log is configured to *not* fsync() on all Append() calls
I20260620 20:09:31.225919 14781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3aa6eb33b35b41d9ad61af761a8d8c69: No bootstrap required, opened a new log
I20260620 20:09:31.229499 14781 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3aa6eb33b35b41d9ad61af761a8d8c69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3aa6eb33b35b41d9ad61af761a8d8c69" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41591 } }
I20260620 20:09:31.229732 14781 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3aa6eb33b35b41d9ad61af761a8d8c69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:31.229782 14781 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3aa6eb33b35b41d9ad61af761a8d8c69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aa6eb33b35b41d9ad61af761a8d8c69, State: Initialized, Role: FOLLOWER
I20260620 20:09:31.230217 14781 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3aa6eb33b35b41d9ad61af761a8d8c69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3aa6eb33b35b41d9ad61af761a8d8c69" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41591 } }
I20260620 20:09:31.230358 14781 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3aa6eb33b35b41d9ad61af761a8d8c69 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:09:31.230425 14781 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3aa6eb33b35b41d9ad61af761a8d8c69 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:09:31.230518 14781 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3aa6eb33b35b41d9ad61af761a8d8c69 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:31.231403 14781 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3aa6eb33b35b41d9ad61af761a8d8c69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3aa6eb33b35b41d9ad61af761a8d8c69" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41591 } }
I20260620 20:09:31.231714 14781 leader_election.cc:302] T 00000000000000000000000000000000 P 3aa6eb33b35b41d9ad61af761a8d8c69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3aa6eb33b35b41d9ad61af761a8d8c69; no voters:
I20260620 20:09:31.232012 14781 leader_election.cc:288] T 00000000000000000000000000000000 P 3aa6eb33b35b41d9ad61af761a8d8c69 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:09:31.232110 14786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3aa6eb33b35b41d9ad61af761a8d8c69 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:09:31.232280 14786 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3aa6eb33b35b41d9ad61af761a8d8c69 [term 1 LEADER]: Becoming Leader. State: Replica: 3aa6eb33b35b41d9ad61af761a8d8c69, State: Running, Role: LEADER
I20260620 20:09:31.232632 14786 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3aa6eb33b35b41d9ad61af761a8d8c69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3aa6eb33b35b41d9ad61af761a8d8c69" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41591 } }
I20260620 20:09:31.233299 14781 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3aa6eb33b35b41d9ad61af761a8d8c69 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:09:31.234236 14787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3aa6eb33b35b41d9ad61af761a8d8c69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3aa6eb33b35b41d9ad61af761a8d8c69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3aa6eb33b35b41d9ad61af761a8d8c69" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41591 } } }
I20260620 20:09:31.234356 14787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3aa6eb33b35b41d9ad61af761a8d8c69 [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:31.234747 14795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:09:31.234792 14788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3aa6eb33b35b41d9ad61af761a8d8c69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3aa6eb33b35b41d9ad61af761a8d8c69. Latest consensus state: current_term: 1 leader_uuid: "3aa6eb33b35b41d9ad61af761a8d8c69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3aa6eb33b35b41d9ad61af761a8d8c69" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41591 } } }
I20260620 20:09:31.234963 14788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3aa6eb33b35b41d9ad61af761a8d8c69 [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:31.237948 14795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:09:31.243228 14795 catalog_manager.cc:1347] Generated new cluster ID: 2f45c32d4931413896034f9141b2b5d9
I20260620 20:09:31.243299 14795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:09:31.259514 14795 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:09:31.260855 14795 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:09:31.273036 14795 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3aa6eb33b35b41d9ad61af761a8d8c69: Generated new TSK 0
I20260620 20:09:31.273965 14795 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:09:31.278280 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41591
--builtin_ntp_servers=127.5.91.148:43219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260620 20:09:31.394701 14805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:31.395020 14805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:31.395078 14805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:31.400600 14805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:31.400714 14805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:31.400804 14805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:09:31.407370 14805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:43219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:31.408492 14805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:31.409839 14805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:31.419759 14811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:31.419762 14813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:31.419761 14810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:31.420087 14805 server_base.cc:1034] running on GCE node
I20260620 20:09:31.420562 14805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:31.421217 14805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:31.422477 14805 hybrid_clock.cc:648] HybridClock initialized: now 1781986171422445 us; error 70 us; skew 500 ppm
I20260620 20:09:31.424539 14805 webserver.cc:460] Webserver started at http://127.5.91.129:38501/ using document root <none> and password file <none>
I20260620 20:09:31.425117 14805 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:31.425186 14805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:31.425395 14805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:31.427682 14805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "fdeae0fe80924df9991999ff4dcea6f2"
format_stamp: "Formatted at 2026-06-20 20:09:31 on dist-test-slave-l8h4"
server_key: "d439b45ee27ab6bb0944113151dadf45"
server_key_iv: "2f000d65746341fc0f9426a8692d1a27"
server_key_version: "encryptionkey@0"
I20260620 20:09:31.428224 14805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "fdeae0fe80924df9991999ff4dcea6f2"
format_stamp: "Formatted at 2026-06-20 20:09:31 on dist-test-slave-l8h4"
server_key: "d439b45ee27ab6bb0944113151dadf45"
server_key_iv: "2f000d65746341fc0f9426a8692d1a27"
server_key_version: "encryptionkey@0"
I20260620 20:09:31.433141 14805 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 20:09:31.436300 14819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:31.437716 14805 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 20:09:31.437842 14805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "fdeae0fe80924df9991999ff4dcea6f2"
format_stamp: "Formatted at 2026-06-20 20:09:31 on dist-test-slave-l8h4"
server_key: "d439b45ee27ab6bb0944113151dadf45"
server_key_iv: "2f000d65746341fc0f9426a8692d1a27"
server_key_version: "encryptionkey@0"
I20260620 20:09:31.437999 14805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:31.457527 14805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:31.458225 14805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:31.458387 14805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:31.458936 14805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:31.460233 14805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:31.460342 14805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:31.460420 14805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:31.460445 14805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:31.469224 14805 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:35599
I20260620 20:09:31.469301 14932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:35599 every 8 connection(s)
I20260620 20:09:31.470165 14805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:09:31.476580 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 14805
I20260620 20:09:31.476694 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:09:31.476999 5486 external_mini_cluster.cc:1428] Setting key fe139e74c8509c91236e3b1b7bf0f56f
I20260620 20:09:31.478806 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41591
--builtin_ntp_servers=127.5.91.148:43219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260620 20:09:31.481142 14784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:31.472066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:54335 (local address 127.5.91.190:41591)
0620 20:09:31.472234 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:31.472239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:31.473342 (+ 1103us) server_negotiation.cc:407] Connection header received
0620 20:09:31.474818 (+ 1476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:31.474821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:31.475062 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:31.475203 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:31.476766 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:31.477554 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:31.478894 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:31.479193 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:31.479298 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:31.480558 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:31.480561 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:31.480563 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:09:31.480575 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:31.480593 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:09:31.480601 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:31.480660 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:31.480871 (+ 211us) server_negotiation.cc:299] Negotiation successful
0620 20:09:31.480940 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260620 20:09:31.482106 14935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:31.472450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41591 (local address 127.5.91.129:54335)
0620 20:09:31.473206 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:31.473250 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:31.474600 (+ 1350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:31.475371 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:31.475377 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:31.475648 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:31.476442 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:31.476464 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:31.477690 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:31.477694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:31.478768 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:31.478780 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:31.479381 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:31.479384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:31.479543 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:31.480213 (+ 670us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 20:09:31.480215 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:31.480278 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 20:09:31.480379 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:31.480674 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:31.480681 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:31.480687 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 20:09:31.480808 (+ 121us) client_negotiation.cc:240] Negotiation successful
0620 20:09:31.481129 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":113,"threads_started":1}
I20260620 20:09:31.483371 14933 heartbeater.cc:347] Connected to a master server at 127.5.91.190:41591
I20260620 20:09:31.483704 14933 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:31.484400 14933 heartbeater.cc:511] Master 127.5.91.190:41591 requested a full tablet report, sending...
I20260620 20:09:31.486163 14744 ts_manager.cc:194] Registered new tserver with Master: fdeae0fe80924df9991999ff4dcea6f2 (127.5.91.129:35599)
I20260620 20:09:31.487752 14744 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.91.129:54335
W20260620 20:09:31.589192 14936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:31.589464 14936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:31.589502 14936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:31.593679 14936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:31.593751 14936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:31.593837 14936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:09:31.598978 14936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:43219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:31.600070 14936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:31.601831 14936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:31.612859 14942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:31.612951 14936 server_base.cc:1034] running on GCE node
W20260620 20:09:31.612869 14944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:31.612859 14941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:31.613744 14936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:31.614480 14936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:31.615659 14936 hybrid_clock.cc:648] HybridClock initialized: now 1781986171615642 us; error 33 us; skew 500 ppm
I20260620 20:09:31.617611 14936 webserver.cc:460] Webserver started at http://127.5.91.130:33449/ using document root <none> and password file <none>
I20260620 20:09:31.618609 14936 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:31.618685 14936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:31.618863 14936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:31.621009 14936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "410ac9825c8b49f1af3396182176ed5d"
format_stamp: "Formatted at 2026-06-20 20:09:31 on dist-test-slave-l8h4"
server_key: "1d480360c65ea0f5a9799b8c4493b529"
server_key_iv: "c532255796ca139f9c2f846f1dc10375"
server_key_version: "encryptionkey@0"
I20260620 20:09:31.621552 14936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "410ac9825c8b49f1af3396182176ed5d"
format_stamp: "Formatted at 2026-06-20 20:09:31 on dist-test-slave-l8h4"
server_key: "1d480360c65ea0f5a9799b8c4493b529"
server_key_iv: "c532255796ca139f9c2f846f1dc10375"
server_key_version: "encryptionkey@0"
I20260620 20:09:31.625216 14936 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260620 20:09:31.627976 14950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:31.629470 14936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 20:09:31.629583 14936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "410ac9825c8b49f1af3396182176ed5d"
format_stamp: "Formatted at 2026-06-20 20:09:31 on dist-test-slave-l8h4"
server_key: "1d480360c65ea0f5a9799b8c4493b529"
server_key_iv: "c532255796ca139f9c2f846f1dc10375"
server_key_version: "encryptionkey@0"
I20260620 20:09:31.629707 14936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:31.655525 14936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:31.656210 14936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:31.656378 14936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:31.656942 14936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:31.657981 14936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:31.658038 14936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:31.658090 14936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:31.658120 14936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:31.666118 14936 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:39651
I20260620 20:09:31.666183 15063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:39651 every 8 connection(s)
I20260620 20:09:31.667004 14936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:09:31.676498 14784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:31.668784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:54009 (local address 127.5.91.190:41591)
0620 20:09:31.668938 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:31.668943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:31.669916 (+ 973us) server_negotiation.cc:407] Connection header received
0620 20:09:31.671373 (+ 1457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:31.671376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:31.671422 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:31.671529 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:31.672484 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:31.673084 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:31.674377 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:31.674598 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:31.674690 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:31.675793 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:31.675796 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:31.675798 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:09:31.675812 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:31.675831 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:09:31.675840 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:31.675906 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:31.676187 (+ 281us) server_negotiation.cc:299] Negotiation successful
0620 20:09:31.676301 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 20:09:31.676651 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 14936
I20260620 20:09:31.676776 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:09:31.677024 15066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:31.669163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41591 (local address 127.5.91.130:54009)
0620 20:09:31.669773 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:31.669813 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:31.671181 (+ 1368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:31.671717 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:31.671724 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:31.671810 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:31.672298 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:31.672312 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:31.673191 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:31.673195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:31.674263 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:31.674273 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:31.674753 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:31.674756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:31.674906 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:31.675517 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 20:09:31.675519 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:31.675555 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 20:09:31.675616 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:31.675914 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:31.675919 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:31.675922 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 20:09:31.676018 (+ 96us) client_negotiation.cc:240] Negotiation successful
0620 20:09:31.676230 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":103,"threads_started":1}
I20260620 20:09:31.677135 5486 external_mini_cluster.cc:1428] Setting key 3762294aec748adf8353b1a66eb99f03
I20260620 20:09:31.678354 15064 heartbeater.cc:347] Connected to a master server at 127.5.91.190:41591
I20260620 20:09:31.678589 15064 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:31.679072 15064 heartbeater.cc:511] Master 127.5.91.190:41591 requested a full tablet report, sending...
I20260620 20:09:31.679317 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41591
--builtin_ntp_servers=127.5.91.148:43219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260620 20:09:31.680383 14744 ts_manager.cc:194] Registered new tserver with Master: 410ac9825c8b49f1af3396182176ed5d (127.5.91.130:39651)
I20260620 20:09:31.681151 14744 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.91.130:54009
W20260620 20:09:31.796311 15067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:31.796553 15067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:31.796605 15067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:31.800631 15067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:31.800688 15067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:31.800761 15067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:09:31.805397 15067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:43219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:31.806494 15067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:31.807869 15067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:31.817152 15075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:31.817157 15073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:31.817149 15072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:31.817426 15067 server_base.cc:1034] running on GCE node
I20260620 20:09:31.817906 15067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:31.818522 15067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:31.819706 15067 hybrid_clock.cc:648] HybridClock initialized: now 1781986171819673 us; error 33 us; skew 500 ppm
I20260620 20:09:31.821485 15067 webserver.cc:460] Webserver started at http://127.5.91.131:33719/ using document root <none> and password file <none>
I20260620 20:09:31.822039 15067 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:31.822105 15067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:31.822321 15067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:31.824311 15067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "8d87c5ca77664d0aa525fb05e855bf2f"
format_stamp: "Formatted at 2026-06-20 20:09:31 on dist-test-slave-l8h4"
server_key: "8b41c4afdc9199e07b7afa16404d396d"
server_key_iv: "477f872f3e683e35fd5ff50c81aad36e"
server_key_version: "encryptionkey@0"
I20260620 20:09:31.824831 15067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "8d87c5ca77664d0aa525fb05e855bf2f"
format_stamp: "Formatted at 2026-06-20 20:09:31 on dist-test-slave-l8h4"
server_key: "8b41c4afdc9199e07b7afa16404d396d"
server_key_iv: "477f872f3e683e35fd5ff50c81aad36e"
server_key_version: "encryptionkey@0"
I20260620 20:09:31.828333 15067 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260620 20:09:31.830838 15081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:31.831998 15067 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260620 20:09:31.832101 15067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "8d87c5ca77664d0aa525fb05e855bf2f"
format_stamp: "Formatted at 2026-06-20 20:09:31 on dist-test-slave-l8h4"
server_key: "8b41c4afdc9199e07b7afa16404d396d"
server_key_iv: "477f872f3e683e35fd5ff50c81aad36e"
server_key_version: "encryptionkey@0"
I20260620 20:09:31.832194 15067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:31.850747 15067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:31.851410 15067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:31.851586 15067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:31.852154 15067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:31.853036 15067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:31.853088 15067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:31.853138 15067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:31.853168 15067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:31.862661 15067 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:45583
I20260620 20:09:31.862675 15194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:45583 every 8 connection(s)
I20260620 20:09:31.863657 15067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:09:31.866937 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 15067
I20260620 20:09:31.867033 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:09:31.867322 5486 external_mini_cluster.cc:1428] Setting key a16bee85f6bbb3ca5150d03c6a671347
I20260620 20:09:31.873147 14784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:31.865573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:57781 (local address 127.5.91.190:41591)
0620 20:09:31.865717 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:31.865721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:31.866724 (+ 1003us) server_negotiation.cc:407] Connection header received
0620 20:09:31.868602 (+ 1878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:31.868605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:31.868662 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:31.868736 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:31.869623 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:31.870176 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:31.871288 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:31.871506 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:31.871591 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:31.872648 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:31.872651 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:31.872655 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:09:31.872667 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:31.872683 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:09:31.872711 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:31.872767 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:31.872929 (+ 162us) server_negotiation.cc:299] Negotiation successful
0620 20:09:31.872992 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 20:09:31.874034 15197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:31.865923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41591 (local address 127.5.91.131:57781)
0620 20:09:31.866565 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:31.866625 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:31.868210 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:31.868911 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:31.868919 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:31.869009 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:31.869484 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:31.869504 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:31.870283 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:31.870285 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:31.871175 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:31.871185 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:31.871639 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:31.871644 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:31.871794 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:31.872392 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 20:09:31.872393 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:31.872434 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 20:09:31.872506 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:31.872780 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:31.872788 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:31.872794 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 20:09:31.872921 (+ 127us) client_negotiation.cc:240] Negotiation successful
0620 20:09:31.873269 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":104,"threads_started":1}
I20260620 20:09:31.875114 15195 heartbeater.cc:347] Connected to a master server at 127.5.91.190:41591
I20260620 20:09:31.875344 15195 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:31.875857 15195 heartbeater.cc:511] Master 127.5.91.190:41591 requested a full tablet report, sending...
I20260620 20:09:31.877048 14744 ts_manager.cc:194] Registered new tserver with Master: 8d87c5ca77664d0aa525fb05e855bf2f (127.5.91.131:45583)
I20260620 20:09:31.877593 14744 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.91.131:57781
I20260620 20:09:31.881306 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 20:09:31.884279 14784 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:31.883412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42426 (local address 127.5.91.190:41591)
0620 20:09:31.883562 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:31.883567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:31.883731 (+ 164us) server_negotiation.cc:407] Connection header received
0620 20:09:31.883796 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:31.883798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:31.883840 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:31.883903 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:31.884140 (+ 237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42426: BlockingRecv error: recv got EOF from 127.0.0.1:42426 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260620 20:09:31.884333 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 14805
I20260620 20:09:31.889065 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 14936
I20260620 20:09:31.893843 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 15067
I20260620 20:09:31.898043 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 14714
2026-06-20T20:09:31Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (923 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-20T20:09:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:09:31Z Disabled control of system clock
I20260620 20:09:31.927075 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:46831
--webserver_interface=127.5.91.190
--webserver_port=0
--builtin_ntp_servers=127.5.91.148:39035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:46831
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 20:09:32.038717 15206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:32.038985 15206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:32.039022 15206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:32.043079 15206 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:09:32.043138 15206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:32.043152 15206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:32.043169 15206 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:09:32.043182 15206 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:09:32.047909 15206 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:39035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:46831
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:46831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:32.048959 15206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:32.050297 15206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:32.058543 15214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:32.058478 15211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:32.058480 15212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:32.058673 15206 server_base.cc:1034] running on GCE node
I20260620 20:09:32.059281 15206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:32.060323 15206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:32.061517 15206 hybrid_clock.cc:648] HybridClock initialized: now 1781986172061501 us; error 30 us; skew 500 ppm
I20260620 20:09:32.063406 15206 webserver.cc:460] Webserver started at http://127.5.91.190:33937/ using document root <none> and password file <none>
I20260620 20:09:32.063952 15206 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:32.064015 15206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:32.064210 15206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:32.066205 15206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "adce6138a6104631afa515e744044b0a"
format_stamp: "Formatted at 2026-06-20 20:09:32 on dist-test-slave-l8h4"
server_key: "621f09f6fe297800c7701f8c321b9a91"
server_key_iv: "d524b7f00646164d0bdf3b8df0e3cee9"
server_key_version: "encryptionkey@0"
I20260620 20:09:32.066735 15206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "adce6138a6104631afa515e744044b0a"
format_stamp: "Formatted at 2026-06-20 20:09:32 on dist-test-slave-l8h4"
server_key: "621f09f6fe297800c7701f8c321b9a91"
server_key_iv: "d524b7f00646164d0bdf3b8df0e3cee9"
server_key_version: "encryptionkey@0"
I20260620 20:09:32.070364 15206 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260620 20:09:32.072839 15220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:32.074124 15206 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 20:09:32.074292 15206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "adce6138a6104631afa515e744044b0a"
format_stamp: "Formatted at 2026-06-20 20:09:32 on dist-test-slave-l8h4"
server_key: "621f09f6fe297800c7701f8c321b9a91"
server_key_iv: "d524b7f00646164d0bdf3b8df0e3cee9"
server_key_version: "encryptionkey@0"
I20260620 20:09:32.074393 15206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:32.078795 15206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:32.079366 15206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:32.079535 15206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:32.086748 15206 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:46831
I20260620 20:09:32.086766 15272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:46831 every 8 connection(s)
I20260620 20:09:32.087661 15206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:09:32.090732 15273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:32.093185 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 15206
I20260620 20:09:32.093305 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:09:32.093565 5486 external_mini_cluster.cc:1428] Setting key 483523dcd403522aed5a35a61831b0bb
I20260620 20:09:32.096904 15273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adce6138a6104631afa515e744044b0a: Bootstrap starting.
I20260620 20:09:32.099509 15273 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adce6138a6104631afa515e744044b0a: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:32.099634 15276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:32.094876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60598 (local address 127.5.91.190:46831)
0620 20:09:32.095632 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:32.095652 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:32.095693 (+ 41us) server_negotiation.cc:407] Connection header received
0620 20:09:32.096555 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:32.096576 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:32.097013 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:32.097362 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:32.098172 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:32.098179 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:32.098196 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:09:32.098208 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:32.098272 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:09:32.098294 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:32.098424 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:32.098485 (+ 61us) server_negotiation.cc:299] Negotiation successful
0620 20:09:32.098712 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":130,"threads_started":1}
I20260620 20:09:32.100274 15273 log.cc:824] T 00000000000000000000000000000000 P adce6138a6104631afa515e744044b0a: Log is configured to *not* fsync() on all Append() calls
I20260620 20:09:32.102177 15273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adce6138a6104631afa515e744044b0a: No bootstrap required, opened a new log
I20260620 20:09:32.105417 15273 raft_consensus.cc:348] T 00000000000000000000000000000000 P adce6138a6104631afa515e744044b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adce6138a6104631afa515e744044b0a" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46831 } }
I20260620 20:09:32.105639 15273 raft_consensus.cc:374] T 00000000000000000000000000000000 P adce6138a6104631afa515e744044b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:32.105695 15273 raft_consensus.cc:729] T 00000000000000000000000000000000 P adce6138a6104631afa515e744044b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adce6138a6104631afa515e744044b0a, State: Initialized, Role: FOLLOWER
I20260620 20:09:32.106081 15273 consensus_queue.cc:260] T 00000000000000000000000000000000 P adce6138a6104631afa515e744044b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adce6138a6104631afa515e744044b0a" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46831 } }
I20260620 20:09:32.106236 15273 raft_consensus.cc:388] T 00000000000000000000000000000000 P adce6138a6104631afa515e744044b0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:09:32.106310 15273 raft_consensus.cc:482] T 00000000000000000000000000000000 P adce6138a6104631afa515e744044b0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:09:32.106446 15273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adce6138a6104631afa515e744044b0a [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:32.107296 15273 raft_consensus.cc:504] T 00000000000000000000000000000000 P adce6138a6104631afa515e744044b0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adce6138a6104631afa515e744044b0a" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46831 } }
I20260620 20:09:32.107604 15273 leader_election.cc:302] T 00000000000000000000000000000000 P adce6138a6104631afa515e744044b0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adce6138a6104631afa515e744044b0a; no voters:
I20260620 20:09:32.107882 15273 leader_election.cc:288] T 00000000000000000000000000000000 P adce6138a6104631afa515e744044b0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:09:32.108050 15278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adce6138a6104631afa515e744044b0a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:09:32.108238 15278 raft_consensus.cc:686] T 00000000000000000000000000000000 P adce6138a6104631afa515e744044b0a [term 1 LEADER]: Becoming Leader. State: Replica: adce6138a6104631afa515e744044b0a, State: Running, Role: LEADER
I20260620 20:09:32.108528 15278 consensus_queue.cc:237] T 00000000000000000000000000000000 P adce6138a6104631afa515e744044b0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adce6138a6104631afa515e744044b0a" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46831 } }
I20260620 20:09:32.109081 15273 sys_catalog.cc:564] T 00000000000000000000000000000000 P adce6138a6104631afa515e744044b0a [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:09:32.109881 15280 sys_catalog.cc:455] T 00000000000000000000000000000000 P adce6138a6104631afa515e744044b0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader adce6138a6104631afa515e744044b0a. Latest consensus state: current_term: 1 leader_uuid: "adce6138a6104631afa515e744044b0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adce6138a6104631afa515e744044b0a" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46831 } } }
I20260620 20:09:32.110015 15280 sys_catalog.cc:458] T 00000000000000000000000000000000 P adce6138a6104631afa515e744044b0a [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:32.110345 15279 sys_catalog.cc:455] T 00000000000000000000000000000000 P adce6138a6104631afa515e744044b0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "adce6138a6104631afa515e744044b0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adce6138a6104631afa515e744044b0a" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46831 } } }
I20260620 20:09:32.110433 15279 sys_catalog.cc:458] T 00000000000000000000000000000000 P adce6138a6104631afa515e744044b0a [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:32.111212 15287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:09:32.113736 15287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:09:32.119197 15287 catalog_manager.cc:1347] Generated new cluster ID: 3964bd99339a48e080a45713741a1773
I20260620 20:09:32.119261 15287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:09:32.131074 15287 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:09:32.131419 15287 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:09:32.136447 15287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P adce6138a6104631afa515e744044b0a: Generated new TSK 0
I20260620 20:09:32.137246 15287 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:09:32.144974 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46831
--builtin_ntp_servers=127.5.91.148:39035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 20:09:32.252067 15297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:32.252318 15297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:32.252374 15297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:32.256392 15297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:32.256443 15297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:32.256521 15297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:09:32.262271 15297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:39035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:32.263341 15297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:32.264710 15297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:32.274039 15302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:32.274044 15303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:32.274129 15297 server_base.cc:1034] running on GCE node
W20260620 20:09:32.274044 15305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:32.274623 15297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:32.275214 15297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:32.276410 15297 hybrid_clock.cc:648] HybridClock initialized: now 1781986172276385 us; error 36 us; skew 500 ppm
I20260620 20:09:32.278455 15297 webserver.cc:460] Webserver started at http://127.5.91.129:34201/ using document root <none> and password file <none>
I20260620 20:09:32.279031 15297 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:32.279098 15297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:32.279273 15297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:32.281248 15297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "d40d1422d0f842709b424872509b8ca9"
format_stamp: "Formatted at 2026-06-20 20:09:32 on dist-test-slave-l8h4"
server_key: "586529978a0b341bbf9a95ab0da341cc"
server_key_iv: "8969fe2450e85300cae6430fc9933f93"
server_key_version: "encryptionkey@0"
I20260620 20:09:32.281774 15297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "d40d1422d0f842709b424872509b8ca9"
format_stamp: "Formatted at 2026-06-20 20:09:32 on dist-test-slave-l8h4"
server_key: "586529978a0b341bbf9a95ab0da341cc"
server_key_iv: "8969fe2450e85300cae6430fc9933f93"
server_key_version: "encryptionkey@0"
I20260620 20:09:32.285297 15297 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 20:09:32.287801 15311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:32.289031 15297 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 20:09:32.289149 15297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "d40d1422d0f842709b424872509b8ca9"
format_stamp: "Formatted at 2026-06-20 20:09:32 on dist-test-slave-l8h4"
server_key: "586529978a0b341bbf9a95ab0da341cc"
server_key_iv: "8969fe2450e85300cae6430fc9933f93"
server_key_version: "encryptionkey@0"
I20260620 20:09:32.289253 15297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:32.293607 15297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:32.294217 15297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:32.294366 15297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:32.294865 15297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:32.295747 15297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:32.295792 15297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:32.295841 15297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:32.295871 15297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:32.304603 15297 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:40435
I20260620 20:09:32.304639 15424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:40435 every 8 connection(s)
I20260620 20:09:32.305444 15297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:09:32.311195 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 15297
I20260620 20:09:32.311295 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:09:32.311312 15276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:32.307257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:33005 (local address 127.5.91.190:46831)
0620 20:09:32.307390 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:32.307394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:32.308412 (+ 1018us) server_negotiation.cc:407] Connection header received
0620 20:09:32.309711 (+ 1299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:32.309714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:32.309752 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:32.309836 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:32.310873 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:32.310876 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:32.310878 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:09:32.310890 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:32.310909 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:09:32.310917 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:32.311014 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:32.311098 (+ 84us) server_negotiation.cc:299] Negotiation successful
0620 20:09:32.311144 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 20:09:32.311610 5486 external_mini_cluster.cc:1428] Setting key 724f03bda0211e3195b0bf8127896be6
I20260620 20:09:32.312124 15427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:32.307666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46831 (local address 127.5.91.129:33005)
0620 20:09:32.308273 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:32.308310 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:32.309516 (+ 1206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:32.309968 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:32.309974 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:32.310082 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:32.310644 (+ 562us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 20:09:32.310646 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:32.310682 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 20:09:32.310744 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:32.310996 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:32.311000 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:32.311004 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 20:09:32.311110 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 20:09:32.311326 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":102,"threads_started":1}
I20260620 20:09:32.313325 15425 heartbeater.cc:347] Connected to a master server at 127.5.91.190:46831
I20260620 20:09:32.313563 15425 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:32.313602 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46831
--builtin_ntp_servers=127.5.91.148:39035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260620 20:09:32.313992 15425 heartbeater.cc:511] Master 127.5.91.190:46831 requested a full tablet report, sending...
I20260620 20:09:32.315784 15236 ts_manager.cc:194] Registered new tserver with Master: d40d1422d0f842709b424872509b8ca9 (127.5.91.129:40435)
W20260620 20:09:32.428501 15428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:32.428764 15428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:32.428812 15428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:32.432924 15428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:32.433000 15428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:32.433059 15428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:09:32.437809 15428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:39035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:32.438861 15428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:32.440181 15428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:32.449765 15433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:32.449731 15436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:32.449978 15428 server_base.cc:1034] running on GCE node
W20260620 20:09:32.449774 15434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:32.450503 15428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:32.451242 15428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:32.452426 15428 hybrid_clock.cc:648] HybridClock initialized: now 1781986172452380 us; error 62 us; skew 500 ppm
I20260620 20:09:32.454396 15428 webserver.cc:460] Webserver started at http://127.5.91.130:37367/ using document root <none> and password file <none>
I20260620 20:09:32.454944 15428 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:32.455014 15428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:32.455227 15428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:32.457231 15428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "6cdd21b0b0f242a6abed806770d435f4"
format_stamp: "Formatted at 2026-06-20 20:09:32 on dist-test-slave-l8h4"
server_key: "16fbe0080d1a1270c25b52db7c6ff8ee"
server_key_iv: "a44b502191687a805c5294fee6341076"
server_key_version: "encryptionkey@0"
I20260620 20:09:32.457716 15428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "6cdd21b0b0f242a6abed806770d435f4"
format_stamp: "Formatted at 2026-06-20 20:09:32 on dist-test-slave-l8h4"
server_key: "16fbe0080d1a1270c25b52db7c6ff8ee"
server_key_iv: "a44b502191687a805c5294fee6341076"
server_key_version: "encryptionkey@0"
I20260620 20:09:32.461313 15428 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 20:09:32.463807 15442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:32.465057 15428 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260620 20:09:32.465166 15428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "6cdd21b0b0f242a6abed806770d435f4"
format_stamp: "Formatted at 2026-06-20 20:09:32 on dist-test-slave-l8h4"
server_key: "16fbe0080d1a1270c25b52db7c6ff8ee"
server_key_iv: "a44b502191687a805c5294fee6341076"
server_key_version: "encryptionkey@0"
I20260620 20:09:32.465272 15428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:32.469839 15428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:32.470602 15428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:32.470784 15428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:32.471333 15428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:32.472239 15428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:32.472288 15428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:32.472344 15428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:32.472376 15428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:32.482957 15555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:38453 every 8 connection(s)
I20260620 20:09:32.482982 15428 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:38453
I20260620 20:09:32.484006 15428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:09:32.490068 15276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:32.485935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:50017 (local address 127.5.91.190:46831)
0620 20:09:32.486064 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:32.486067 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:32.487118 (+ 1051us) server_negotiation.cc:407] Connection header received
0620 20:09:32.488427 (+ 1309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:32.488431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:32.488483 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:32.488557 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:32.489598 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:32.489602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:32.489605 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:09:32.489619 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:32.489638 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:09:32.489645 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:32.489723 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:32.489864 (+ 141us) server_negotiation.cc:299] Negotiation successful
0620 20:09:32.489914 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 20:09:32.490922 15558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:32.486285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46831 (local address 127.5.91.130:50017)
0620 20:09:32.486970 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:32.487010 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:32.488210 (+ 1200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:32.488726 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:32.488732 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:32.488819 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:32.489333 (+ 514us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 20:09:32.489334 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:32.489373 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 20:09:32.489444 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:32.489715 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:32.489722 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:32.489728 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 20:09:32.489927 (+ 199us) client_negotiation.cc:240] Negotiation successful
0620 20:09:32.490196 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":106,"threads_started":1}
I20260620 20:09:32.491504 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 15428
I20260620 20:09:32.491606 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:09:32.491868 5486 external_mini_cluster.cc:1428] Setting key 3cd1ca222730385ae87178f15645d2c4
I20260620 20:09:32.492136 15556 heartbeater.cc:347] Connected to a master server at 127.5.91.190:46831
I20260620 20:09:32.492370 15556 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:32.492897 15556 heartbeater.cc:511] Master 127.5.91.190:46831 requested a full tablet report, sending...
I20260620 20:09:32.493994 15236 ts_manager.cc:194] Registered new tserver with Master: 6cdd21b0b0f242a6abed806770d435f4 (127.5.91.130:38453)
I20260620 20:09:32.494076 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46831
--builtin_ntp_servers=127.5.91.148:39035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 20:09:32.605808 15559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:32.606077 15559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:32.606115 15559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:32.610261 15559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:32.610321 15559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:32.610405 15559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:09:32.615749 15559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:39035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:32.616918 15559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:32.618530 15559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:32.628186 15564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:32.628194 15565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:32.628271 15567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:32.628377 15559 server_base.cc:1034] running on GCE node
I20260620 20:09:32.628952 15559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:32.629590 15559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:32.630755 15559 hybrid_clock.cc:648] HybridClock initialized: now 1781986172630720 us; error 52 us; skew 500 ppm
I20260620 20:09:32.632786 15559 webserver.cc:460] Webserver started at http://127.5.91.131:35263/ using document root <none> and password file <none>
I20260620 20:09:32.633325 15559 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:32.633388 15559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:32.633591 15559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:32.635867 15559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "da72ae0f3db6453288c3f8f2a11846e1"
format_stamp: "Formatted at 2026-06-20 20:09:32 on dist-test-slave-l8h4"
server_key: "df4e6f0ab2557aa0545fa5b58e2079a1"
server_key_iv: "2d9c89ecaaeaa55becde6a4fd40596bd"
server_key_version: "encryptionkey@0"
I20260620 20:09:32.636518 15559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "da72ae0f3db6453288c3f8f2a11846e1"
format_stamp: "Formatted at 2026-06-20 20:09:32 on dist-test-slave-l8h4"
server_key: "df4e6f0ab2557aa0545fa5b58e2079a1"
server_key_iv: "2d9c89ecaaeaa55becde6a4fd40596bd"
server_key_version: "encryptionkey@0"
I20260620 20:09:32.640230 15559 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 20:09:32.642880 15573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:32.644166 15559 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:09:32.644284 15559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "da72ae0f3db6453288c3f8f2a11846e1"
format_stamp: "Formatted at 2026-06-20 20:09:32 on dist-test-slave-l8h4"
server_key: "df4e6f0ab2557aa0545fa5b58e2079a1"
server_key_iv: "2d9c89ecaaeaa55becde6a4fd40596bd"
server_key_version: "encryptionkey@0"
I20260620 20:09:32.644388 15559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:32.648811 15559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:32.649374 15559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:32.649521 15559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:32.650032 15559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:32.650960 15559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:32.651006 15559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:32.651055 15559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:32.651083 15559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:32.659019 15559 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:37807
I20260620 20:09:32.659075 15686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:37807 every 8 connection(s)
I20260620 20:09:32.659868 15559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:09:32.660617 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 15559
I20260620 20:09:32.660712 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:09:32.660972 5486 external_mini_cluster.cc:1428] Setting key f5644520987f508a7e758f9fa40a538b
I20260620 20:09:32.666620 15276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:32.662024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:41593 (local address 127.5.91.190:46831)
0620 20:09:32.662183 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:32.662188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:32.663563 (+ 1375us) server_negotiation.cc:407] Connection header received
0620 20:09:32.664994 (+ 1431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:32.664997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:32.665039 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:32.665109 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:32.666191 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:32.666193 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:32.666195 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:09:32.666206 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:32.666225 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:09:32.666232 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:32.666309 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:32.666432 (+ 123us) server_negotiation.cc:299] Negotiation successful
0620 20:09:32.666479 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 20:09:32.667467 15689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:32.662570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46831 (local address 127.5.91.131:41593)
0620 20:09:32.663430 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:32.663473 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:32.664790 (+ 1317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:32.665260 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:32.665266 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:32.665350 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:32.665905 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 20:09:32.665907 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:32.665949 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 20:09:32.666022 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:32.666305 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:32.666313 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:32.666318 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 20:09:32.666488 (+ 170us) client_negotiation.cc:240] Negotiation successful
0620 20:09:32.666722 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":141,"threads_started":1}
I20260620 20:09:32.668613 15687 heartbeater.cc:347] Connected to a master server at 127.5.91.190:46831
I20260620 20:09:32.668856 15687 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:32.669306 15687 heartbeater.cc:511] Master 127.5.91.190:46831 requested a full tablet report, sending...
I20260620 20:09:32.670320 15236 ts_manager.cc:194] Registered new tserver with Master: da72ae0f3db6453288c3f8f2a11846e1 (127.5.91.131:37807)
I20260620 20:09:32.675066 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 20:09:32.677747 15276 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:09:32.676911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60610 (local address 127.5.91.190:46831)
0620 20:09:32.677056 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:32.677061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:32.677210 (+ 149us) server_negotiation.cc:407] Connection header received
0620 20:09:32.677320 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:32.677323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:32.677357 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:32.677418 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:32.677644 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60610: BlockingRecv error: recv got EOF from 127.0.0.1:60610 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 20:09:32.678076 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 15297
I20260620 20:09:32.683399 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 15428
I20260620 20:09:32.688097 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 15559
I20260620 20:09:32.692411 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 15206
2026-06-20T20:09:32Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (793 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:09:32 dist-test-slave-l8h4 krb5kdc[15697](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:09:32 dist-test-slave-l8h4 krb5kdc[15697](info): set up 2 sockets
Jun 20 20:09:32 dist-test-slave-l8h4 krb5kdc[15697](info): commencing operation
krb5kdc: starting...
W20260620 20:09:33.667786 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.956s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:09:33 dist-test-slave-l8h4 krb5kdc[15697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:09:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:09:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:09:33.794662 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:39251
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:34217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:39251
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:09:33.901856 15710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:33.902115 15710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:33.902186 15710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:33.906164 15710 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:09:33.906222 15710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:33.906240 15710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:33.906255 15710 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:09:33.906270 15710 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:09:33.911568 15710 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:34217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:39251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:39251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:33.912568 15710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:33.913520 15710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 20:09:33.922698 15710 server_base.cc:1034] running on GCE node
W20260620 20:09:33.922577 15716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:33.922550 15715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:33.922571 15718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:33.923638 15710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:33.924723 15710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:33.925890 15710 hybrid_clock.cc:648] HybridClock initialized: now 1781986173925878 us; error 29 us; skew 500 ppm
Jun 20 20:09:33 dist-test-slave-l8h4 krb5kdc[15697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986173, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:33.928577 15710 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:09:33.929528 15710 webserver.cc:460] Webserver started at http://127.5.91.190:33959/ using document root <none> and password file <none>
I20260620 20:09:33.930222 15710 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:33.930311 15710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:33.930568 15710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:33.932883 15710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "8cf6c695975642aba23c08088796c203"
format_stamp: "Formatted at 2026-06-20 20:09:33 on dist-test-slave-l8h4"
server_key: "390e9423876416df5e90e8dc9eae1c9d"
server_key_iv: "cb596ab38d93774535c51032efa7286e"
server_key_version: "encryptionkey@0"
I20260620 20:09:33.933369 15710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "8cf6c695975642aba23c08088796c203"
format_stamp: "Formatted at 2026-06-20 20:09:33 on dist-test-slave-l8h4"
server_key: "390e9423876416df5e90e8dc9eae1c9d"
server_key_iv: "cb596ab38d93774535c51032efa7286e"
server_key_version: "encryptionkey@0"
I20260620 20:09:33.936785 15710 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 20:09:33.939072 15726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:33.940706 15710 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:09:33.940802 15710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "8cf6c695975642aba23c08088796c203"
format_stamp: "Formatted at 2026-06-20 20:09:33 on dist-test-slave-l8h4"
server_key: "390e9423876416df5e90e8dc9eae1c9d"
server_key_iv: "cb596ab38d93774535c51032efa7286e"
server_key_version: "encryptionkey@0"
I20260620 20:09:33.940915 15710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:33.979096 15710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:33.982270 15710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:33.982442 15710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:33.989132 15710 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:39251
I20260620 20:09:33.989383 15779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:39251 every 8 connection(s)
I20260620 20:09:33.990083 15710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:09:33.991439 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 15710
I20260620 20:09:33.991554 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:09:33.991842 5486 external_mini_cluster.cc:1428] Setting key 1324be09ad4e3cf574bac2f6b48436b7
I20260620 20:09:33.993284 15780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:33.998634 15780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203: Bootstrap starting.
Jun 20 20:09:33 dist-test-slave-l8h4 krb5kdc[15697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:09:34.000996 15780 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:34.002249 15780 log.cc:824] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203: Log is configured to *not* fsync() on all Append() calls
I20260620 20:09:34.004949 15780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203: No bootstrap required, opened a new log
I20260620 20:09:34.006197 15783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:33.993362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50614 (local address 127.5.91.190:39251)
0620 20:09:33.993808 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:33.993822 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:33.993856 (+ 34us) server_negotiation.cc:407] Connection header received
0620 20:09:33.994593 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:33.994610 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:33.994918 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:33.995207 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:33.995949 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:33.996939 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:33.997760 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:33.998251 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:33.998400 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:34.001011 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:34.001016 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:34.001029 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:34.001066 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:34.002773 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:34.003258 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:34.003263 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:34.003268 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:34.003410 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:34.003756 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:34.003759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:34.003761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:34.004269 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:34.004448 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:34.004804 (+ 356us) server_negotiation.cc:299] Negotiation successful
0620 20:09:34.005160 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":129,"threads_started":1}
I20260620 20:09:34.008944 15780 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf6c695975642aba23c08088796c203" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39251 } }
I20260620 20:09:34.009323 15780 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:34.009394 15780 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cf6c695975642aba23c08088796c203, State: Initialized, Role: FOLLOWER
I20260620 20:09:34.009780 15780 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf6c695975642aba23c08088796c203" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39251 } }
I20260620 20:09:34.009914 15780 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:09:34.009955 15780 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:09:34.010018 15780 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:34.010957 15780 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf6c695975642aba23c08088796c203" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39251 } }
I20260620 20:09:34.011260 15780 leader_election.cc:302] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cf6c695975642aba23c08088796c203; no voters:
I20260620 20:09:34.011549 15780 leader_election.cc:288] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:09:34.011657 15785 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:09:34.011898 15785 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203 [term 1 LEADER]: Becoming Leader. State: Replica: 8cf6c695975642aba23c08088796c203, State: Running, Role: LEADER
I20260620 20:09:34.012200 15785 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf6c695975642aba23c08088796c203" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39251 } }
I20260620 20:09:34.012778 15780 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:09:34.014006 15786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cf6c695975642aba23c08088796c203" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf6c695975642aba23c08088796c203" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39251 } } }
I20260620 20:09:34.014194 15786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203 [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:34.014012 15787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cf6c695975642aba23c08088796c203. Latest consensus state: current_term: 1 leader_uuid: "8cf6c695975642aba23c08088796c203" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf6c695975642aba23c08088796c203" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 39251 } } }
I20260620 20:09:34.014254 15787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203 [sys.catalog]: This master's current role is: LEADER
I20260620 20:09:34.014600 15794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:09:34.017264 15794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:09:34.022215 15794 catalog_manager.cc:1347] Generated new cluster ID: 1c3e4d56f24944af82751695bbcb53f2
I20260620 20:09:34.022279 15794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:09:34.041069 15794 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:09:34.041994 15794 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:09:34.051249 15794 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203: Generated new TSK 0
I20260620 20:09:34.051944 15794 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.91.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
I20260620 20:09:34.077512 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:39251
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:34217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:09:34.185554 15806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:34.185803 15806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:34.185845 15806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:34.189841 15806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:34.189903 15806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:34.189970 15806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:09:34.195178 15806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:34217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:39251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:34.196162 15806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:34.197117 15806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:34.207010 15811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:34.207010 15814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:34.207383 15812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:34.207631 15806 server_base.cc:1034] running on GCE node
I20260620 20:09:34.208021 15806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:34.208633 15806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:34.209793 15806 hybrid_clock.cc:648] HybridClock initialized: now 1781986174209774 us; error 32 us; skew 500 ppm
Jun 20 20:09:34 dist-test-slave-l8h4 krb5kdc[15697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986174, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:34.212497 15806 init.cc:377] Logged in from keytab as kudu/127.5.91.129@KRBTEST.COM (short username kudu)
I20260620 20:09:34.213459 15806 webserver.cc:460] Webserver started at http://127.5.91.129:36821/ using document root <none> and password file <none>
I20260620 20:09:34.214054 15806 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:34.214116 15806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:34.214330 15806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:34.216552 15806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "597a06fa4de74b659c3e0890b56930c0"
format_stamp: "Formatted at 2026-06-20 20:09:34 on dist-test-slave-l8h4"
server_key: "83c30a46d44f4a684c21264a957e6207"
server_key_iv: "432c046533b7cf9607b8ce4c95b0fb95"
server_key_version: "encryptionkey@0"
I20260620 20:09:34.217142 15806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "597a06fa4de74b659c3e0890b56930c0"
format_stamp: "Formatted at 2026-06-20 20:09:34 on dist-test-slave-l8h4"
server_key: "83c30a46d44f4a684c21264a957e6207"
server_key_iv: "432c046533b7cf9607b8ce4c95b0fb95"
server_key_version: "encryptionkey@0"
I20260620 20:09:34.221314 15806 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260620 20:09:34.223826 15821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:34.225162 15806 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 20:09:34.225263 15806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "597a06fa4de74b659c3e0890b56930c0"
format_stamp: "Formatted at 2026-06-20 20:09:34 on dist-test-slave-l8h4"
server_key: "83c30a46d44f4a684c21264a957e6207"
server_key_iv: "432c046533b7cf9607b8ce4c95b0fb95"
server_key_version: "encryptionkey@0"
I20260620 20:09:34.225370 15806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:34.236449 15806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:34.239265 15806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:34.239437 15806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:34.239974 15806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:34.240955 15806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:34.241009 15806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:34.241053 15806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:34.241077 15806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:34.250392 15806 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:44713
I20260620 20:09:34.250440 15935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:44713 every 8 connection(s)
I20260620 20:09:34.251271 15806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:09:34.253480 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 15806
I20260620 20:09:34.253594 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:09:34.253891 5486 external_mini_cluster.cc:1428] Setting key a9e9206cfe656042660b0c60bf54482d
Jun 20 20:09:34 dist-test-slave-l8h4 krb5kdc[15697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986174, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.91.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.130@KRBTEST.COM" created.
I20260620 20:09:34.268225 15783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:34.253094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:33843 (local address 127.5.91.190:39251)
0620 20:09:34.253293 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:34.253298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:34.254101 (+ 803us) server_negotiation.cc:407] Connection header received
0620 20:09:34.255764 (+ 1663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:34.255767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:34.255811 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:34.255881 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:34.257369 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.258244 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.259849 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.260244 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.260341 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:34.264131 (+ 3790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:34.264135 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:34.264137 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:34.264163 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:34.265853 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:34.266421 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:34.266424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:34.266426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:34.266535 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:34.266962 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:34.266966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:34.266967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:34.267228 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:34.267325 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:34.267873 (+ 548us) server_negotiation.cc:299] Negotiation successful
0620 20:09:34.268040 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260620 20:09:34.269136 15938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:34.253436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:39251 (local address 127.5.91.129:33843)
0620 20:09:34.253945 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:34.254003 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:34.255520 (+ 1517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:34.256071 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:34.256080 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:34.256536 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:34.257196 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:34.257217 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.258406 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.258410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:34.259702 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:34.259714 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.260406 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.260411 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:34.260630 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:34.262097 (+ 1467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:34.262120 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:34.263927 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:34.265996 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:34.266001 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:34.266014 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:34.266331 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:34.266656 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:34.266660 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:34.266662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:34.266848 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:34.267351 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:34.267356 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:34.267632 (+ 276us) client_negotiation.cc:769] Sending connection context
0620 20:09:34.267795 (+ 163us) client_negotiation.cc:240] Negotiation successful
0620 20:09:34.268075 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":133,"threads_started":1}
I20260620 20:09:34.270541 15936 heartbeater.cc:347] Connected to a master server at 127.5.91.190:39251
I20260620 20:09:34.270808 15936 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:34.271306 15936 heartbeater.cc:511] Master 127.5.91.190:39251 requested a full tablet report, sending...
I20260620 20:09:34.272899 15744 ts_manager.cc:194] Registered new tserver with Master: 597a06fa4de74b659c3e0890b56930c0 (127.5.91.129:44713)
I20260620 20:09:34.274001 15744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:33843
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
I20260620 20:09:34.285213 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:39251
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:34217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:09:34.396010 15941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:34.396258 15941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:34.396317 15941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:34.400672 15941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:34.400748 15941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:34.400830 15941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:09:34.406431 15941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:34217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:39251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:34.407723 15941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:34.408748 15941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:34.418288 15947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:34.418349 15949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:34.418346 15946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:34.418411 15941 server_base.cc:1034] running on GCE node
I20260620 20:09:34.418913 15941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:34.419503 15941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:34.420656 15941 hybrid_clock.cc:648] HybridClock initialized: now 1781986174420631 us; error 35 us; skew 500 ppm
Jun 20 20:09:34 dist-test-slave-l8h4 krb5kdc[15697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986174, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:34.423363 15941 init.cc:377] Logged in from keytab as kudu/127.5.91.130@KRBTEST.COM (short username kudu)
I20260620 20:09:34.424221 15941 webserver.cc:460] Webserver started at http://127.5.91.130:45075/ using document root <none> and password file <none>
I20260620 20:09:34.424705 15941 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:34.424767 15941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:34.424921 15941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:34.427013 15941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "a0f650c5f0e8419493db248c439a4b87"
format_stamp: "Formatted at 2026-06-20 20:09:34 on dist-test-slave-l8h4"
server_key: "1d92d483f2a09e938c3d0e10d9d28bba"
server_key_iv: "bec436a882b44dcf2b4995119356c9ef"
server_key_version: "encryptionkey@0"
I20260620 20:09:34.427506 15941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "a0f650c5f0e8419493db248c439a4b87"
format_stamp: "Formatted at 2026-06-20 20:09:34 on dist-test-slave-l8h4"
server_key: "1d92d483f2a09e938c3d0e10d9d28bba"
server_key_iv: "bec436a882b44dcf2b4995119356c9ef"
server_key_version: "encryptionkey@0"
I20260620 20:09:34.431115 15941 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 20:09:34.433498 15956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:34.434775 15941 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:09:34.434867 15941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "a0f650c5f0e8419493db248c439a4b87"
format_stamp: "Formatted at 2026-06-20 20:09:34 on dist-test-slave-l8h4"
server_key: "1d92d483f2a09e938c3d0e10d9d28bba"
server_key_iv: "bec436a882b44dcf2b4995119356c9ef"
server_key_version: "encryptionkey@0"
I20260620 20:09:34.434959 15941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:34.456828 15941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:34.459705 15941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:34.459868 15941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:34.460381 15941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:34.461155 15941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:34.461199 15941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:34.461236 15941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:34.461256 15941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:34.469466 15941 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:45771
I20260620 20:09:34.469497 16070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:45771 every 8 connection(s)
I20260620 20:09:34.470319 15941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:09:34.472489 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 15941
I20260620 20:09:34.472597 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:09:34.472946 5486 external_mini_cluster.cc:1428] Setting key 37b8fea9d88ab4b9a617243af3f8a190
Jun 20 20:09:34 dist-test-slave-l8h4 krb5kdc[15697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986174, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.91.131@KRBTEST.COM; defaulting to no policy
I20260620 20:09:34.485376 15783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:34.472028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:45105 (local address 127.5.91.190:39251)
0620 20:09:34.472182 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:34.472185 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:34.472992 (+ 807us) server_negotiation.cc:407] Connection header received
0620 20:09:34.474491 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:34.474494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:34.474535 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:34.474614 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:34.476036 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.476561 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.477761 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.477999 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.478082 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:34.481097 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:34.481110 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:34.481113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:34.481147 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:34.482707 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:34.483325 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:34.483328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:34.483329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:34.483408 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:34.483890 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:34.483893 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:34.483894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:34.484085 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:34.484162 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:34.485114 (+ 952us) server_negotiation.cc:299] Negotiation successful
0620 20:09:34.485228 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.131@KRBTEST.COM" created.
I20260620 20:09:34.486541 16073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:34.472331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:39251 (local address 127.5.91.130:45105)
0620 20:09:34.472829 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:34.472884 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:34.474285 (+ 1401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:34.474813 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:34.474821 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:34.475278 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:34.475890 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:34.475907 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.476744 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.476747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:34.477629 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:34.477638 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.478179 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.478184 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:34.478365 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:34.478954 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:34.478973 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:34.480810 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:34.482855 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:34.482862 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:34.482879 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:34.483210 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:34.483551 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:34.483554 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:34.483556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:34.483741 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:34.484271 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:34.484279 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:34.484808 (+ 529us) client_negotiation.cc:769] Sending connection context
0620 20:09:34.485016 (+ 208us) client_negotiation.cc:240] Negotiation successful
0620 20:09:34.485351 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":105,"threads_started":1}
I20260620 20:09:34.488732 16071 heartbeater.cc:347] Connected to a master server at 127.5.91.190:39251
I20260620 20:09:34.488966 16071 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:34.489530 16071 heartbeater.cc:511] Master 127.5.91.190:39251 requested a full tablet report, sending...
I20260620 20:09:34.490608 15744 ts_manager.cc:194] Registered new tserver with Master: a0f650c5f0e8419493db248c439a4b87 (127.5.91.130:45771)
I20260620 20:09:34.491158 15744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:45105
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
I20260620 20:09:34.502523 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:39251
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:34217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:09:34.608121 16076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:09:34.608362 16076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:09:34.608404 16076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:09:34.612319 16076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:09:34.612373 16076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:09:34.612434 16076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:09:34.617738 16076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:34217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:39251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:09:34.618724 16076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:09:34.619645 16076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:09:34.628945 16081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:34.628949 16084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:09:34.628959 16082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:09:34.629156 16076 server_base.cc:1034] running on GCE node
I20260620 20:09:34.629578 16076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:09:34.630236 16076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:09:34.631417 16076 hybrid_clock.cc:648] HybridClock initialized: now 1781986174631400 us; error 25 us; skew 500 ppm
Jun 20 20:09:34 dist-test-slave-l8h4 krb5kdc[15697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986174, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:09:34.633867 16076 init.cc:377] Logged in from keytab as kudu/127.5.91.131@KRBTEST.COM (short username kudu)
I20260620 20:09:34.634685 16076 webserver.cc:460] Webserver started at http://127.5.91.131:44691/ using document root <none> and password file <none>
I20260620 20:09:34.635156 16076 fs_manager.cc:359] Metadata directory not provided
I20260620 20:09:34.635208 16076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:09:34.635356 16076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:09:34.637349 16076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "455aeac0af934439bc048191a5560733"
format_stamp: "Formatted at 2026-06-20 20:09:34 on dist-test-slave-l8h4"
server_key: "4e478c4875164ab167874c327081bbf9"
server_key_iv: "51fca28d539682a707a8f6023368f0b5"
server_key_version: "encryptionkey@0"
I20260620 20:09:34.637844 16076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "455aeac0af934439bc048191a5560733"
format_stamp: "Formatted at 2026-06-20 20:09:34 on dist-test-slave-l8h4"
server_key: "4e478c4875164ab167874c327081bbf9"
server_key_iv: "51fca28d539682a707a8f6023368f0b5"
server_key_version: "encryptionkey@0"
I20260620 20:09:34.641465 16076 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 20:09:34.643875 16091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:34.645056 16076 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 20:09:34.645162 16076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "455aeac0af934439bc048191a5560733"
format_stamp: "Formatted at 2026-06-20 20:09:34 on dist-test-slave-l8h4"
server_key: "4e478c4875164ab167874c327081bbf9"
server_key_iv: "51fca28d539682a707a8f6023368f0b5"
server_key_version: "encryptionkey@0"
I20260620 20:09:34.645263 16076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:09:34.672591 16076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:09:34.676393 16076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:09:34.676594 16076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:09:34.677161 16076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:09:34.677956 16076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:09:34.678004 16076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:34.678053 16076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:09:34.678079 16076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:09:34.687631 16076 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:38141
I20260620 20:09:34.687714 16205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:38141 every 8 connection(s)
I20260620 20:09:34.688504 16076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:09:34.688941 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 16076
I20260620 20:09:34.689043 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:09:34.689285 5486 external_mini_cluster.cc:1428] Setting key 646da6625f3c609b4dad66185aab91d3
Jun 20 20:09:34 dist-test-slave-l8h4 krb5kdc[15697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986174, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:09:34.704368 15783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:34.690361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:48559 (local address 127.5.91.190:39251)
0620 20:09:34.690490 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:34.690494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:34.691572 (+ 1078us) server_negotiation.cc:407] Connection header received
0620 20:09:34.692853 (+ 1281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:34.692855 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:34.692909 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:34.692976 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:34.694049 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.694640 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.696176 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.696416 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.696613 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:34.700472 (+ 3859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:34.700475 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:34.700477 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:34.700515 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:34.701994 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:34.702567 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:34.702570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:34.702572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:34.702651 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:34.703091 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:34.703095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:34.703096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:34.703292 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:34.703368 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:34.704116 (+ 748us) server_negotiation.cc:299] Negotiation successful
0620 20:09:34.704235 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 20:09:34.705112 16208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:34.690885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:39251 (local address 127.5.91.131:48559)
0620 20:09:34.691414 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:34.691485 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:34.692625 (+ 1140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:34.693112 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:34.693118 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:34.693463 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:09:34.693904 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:34.693916 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.694756 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.694759 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:34.695967 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:34.695991 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.696953 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.696958 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:34.697187 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:34.697992 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:09:34.698014 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:09:34.700268 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:09:34.702106 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:34.702111 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:34.702124 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:34.702474 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:34.702759 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:09:34.702762 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:09:34.702763 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:09:34.703001 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:09:34.703425 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:09:34.703430 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:09:34.703698 (+ 268us) client_negotiation.cc:769] Sending connection context
0620 20:09:34.703859 (+ 161us) client_negotiation.cc:240] Negotiation successful
0620 20:09:34.704135 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":144,"threads_started":1}
I20260620 20:09:34.706350 16206 heartbeater.cc:347] Connected to a master server at 127.5.91.190:39251
I20260620 20:09:34.706765 16206 heartbeater.cc:464] Registering TS with master...
I20260620 20:09:34.707266 16206 heartbeater.cc:511] Master 127.5.91.190:39251 requested a full tablet report, sending...
I20260620 20:09:34.708472 15744 ts_manager.cc:194] Registered new tserver with Master: 455aeac0af934439bc048191a5560733 (127.5.91.131:38141)
I20260620 20:09:34.709215 15744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:48559
I20260620 20:09:34.715219 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 20:09:34.725404 15783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:34.717448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50630 (local address 127.5.91.190:39251)
0620 20:09:34.717595 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:34.717599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:34.717679 (+ 80us) server_negotiation.cc:407] Connection header received
0620 20:09:34.717802 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:34.717805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:34.717966 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:34.718226 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:09:34.718953 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.719481 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.720304 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.720534 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.720619 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:34.722106 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:09:34.722109 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:09:34.722111 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:09:34.722133 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:09:34.723825 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:34.724248 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:34.724251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:34.724253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:34.724328 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:09:34.724570 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:09:34.724573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:09:34.724575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:09:34.724792 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:09:34.724917 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:34.725136 (+ 219us) server_negotiation.cc:299] Negotiation successful
0620 20:09:34.725249 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260620 20:09:34.728226 15744 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50630:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:09:34.731266 15744 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:09:34.747020 16218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:34.741018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36890 (local address 127.5.91.131:38141)
0620 20:09:34.741936 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:34.741947 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:34.741971 (+ 24us) server_negotiation.cc:407] Connection header received
0620 20:09:34.742042 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:34.742048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:34.742280 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:34.742465 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:34.742767 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.744060 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.745627 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.746345 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.746450 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:34.746702 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:34.746764 (+ 62us) server_negotiation.cc:299] Negotiation successful
0620 20:09:34.746859 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":812,"thread_start_us":175,"threads_started":1}
I20260620 20:09:34.747087 16217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:34.740873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:38141 (local address 127.0.0.1:36890)
0620 20:09:34.741339 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:34.741387 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:34.741505 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:34.742475 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:34.742481 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:34.742505 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:34.742641 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:34.742648 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.744193 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.744196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:34.745531 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:34.745539 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.746470 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.746475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:34.746584 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:34.746626 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 20:09:34.746727 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 20:09:34.746924 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":16,"thread_start_us":93,"threads_started":1}
I20260620 20:09:34.747506 16219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:34.741049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:45771 (local address 127.0.0.1:60898)
0620 20:09:34.741966 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:34.741978 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:34.742080 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:34.742641 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:34.742644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:34.742659 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:34.742768 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:34.742774 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.743971 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.743974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:34.745433 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:34.745443 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.746788 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.746792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:34.746878 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:34.746903 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 20:09:34.746979 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 20:09:34.747043 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":850,"mutex_wait_us":58,"spinlock_wait_cycles":896,"thread_start_us":216,"threads_started":1}
I20260620 20:09:34.747818 16221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:34.741699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:44713 (local address 127.0.0.1:46020)
0620 20:09:34.741995 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:34.742007 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:34.742695 (+ 688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:34.743131 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:34.743134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:34.743152 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:34.743239 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:34.743244 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.744636 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.744640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:34.745858 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:34.745869 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.747474 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.747477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:34.747551 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:34.747576 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 20:09:34.747648 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 20:09:34.747715 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":2,"thread_start_us":76,"threads_started":1}
I20260620 20:09:34.748126 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:34.741480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60898 (local address 127.5.91.130:45771)
0620 20:09:34.741891 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:34.741903 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:34.742080 (+ 177us) server_negotiation.cc:407] Connection header received
0620 20:09:34.742214 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:34.742220 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:34.742400 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:34.742547 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:34.742902 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.743817 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.745656 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.746470 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.746582 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:34.746724 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:34.747765 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0620 20:09:34.747869 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":134,"threads_started":1}
I20260620 20:09:34.748198 16222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:34.741859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46020 (local address 127.5.91.129:44713)
0620 20:09:34.742321 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:34.742335 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:34.742354 (+ 19us) server_negotiation.cc:407] Connection header received
0620 20:09:34.742831 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:34.742837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:34.742998 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:34.743136 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:34.743331 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.744497 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.745990 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.746936 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.747041 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:34.747306 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:34.747976 (+ 670us) server_negotiation.cc:299] Negotiation successful
0620 20:09:34.748069 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":69,"threads_started":1}
I20260620 20:09:34.750370 16140 tablet_service.cc:1467] Processing CreateTablet for tablet f4092ba841454f1f8899ce34a44bfdd6 (DEFAULT_TABLE table=test-table [id=70ab770bd4d542f983390e4cccbd50ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:09:34.750759 16005 tablet_service.cc:1467] Processing CreateTablet for tablet f4092ba841454f1f8899ce34a44bfdd6 (DEFAULT_TABLE table=test-table [id=70ab770bd4d542f983390e4cccbd50ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:09:34.750757 15870 tablet_service.cc:1467] Processing CreateTablet for tablet f4092ba841454f1f8899ce34a44bfdd6 (DEFAULT_TABLE table=test-table [id=70ab770bd4d542f983390e4cccbd50ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:09:34.751258 16140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4092ba841454f1f8899ce34a44bfdd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:34.751744 15870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4092ba841454f1f8899ce34a44bfdd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:34.751751 16005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4092ba841454f1f8899ce34a44bfdd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:09:34.757222 16223 tablet_bootstrap.cc:492] T f4092ba841454f1f8899ce34a44bfdd6 P 455aeac0af934439bc048191a5560733: Bootstrap starting.
I20260620 20:09:34.759383 16225 tablet_bootstrap.cc:492] T f4092ba841454f1f8899ce34a44bfdd6 P 597a06fa4de74b659c3e0890b56930c0: Bootstrap starting.
I20260620 20:09:34.759549 16224 tablet_bootstrap.cc:492] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87: Bootstrap starting.
I20260620 20:09:34.760530 16223 tablet_bootstrap.cc:654] T f4092ba841454f1f8899ce34a44bfdd6 P 455aeac0af934439bc048191a5560733: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:34.761605 16225 tablet_bootstrap.cc:654] T f4092ba841454f1f8899ce34a44bfdd6 P 597a06fa4de74b659c3e0890b56930c0: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:34.762058 16224 tablet_bootstrap.cc:654] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87: Neither blocks nor log segments found. Creating new log.
I20260620 20:09:34.762019 16223 log.cc:824] T f4092ba841454f1f8899ce34a44bfdd6 P 455aeac0af934439bc048191a5560733: Log is configured to *not* fsync() on all Append() calls
I20260620 20:09:34.763005 16224 log.cc:824] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87: Log is configured to *not* fsync() on all Append() calls
I20260620 20:09:34.763005 16225 log.cc:824] T f4092ba841454f1f8899ce34a44bfdd6 P 597a06fa4de74b659c3e0890b56930c0: Log is configured to *not* fsync() on all Append() calls
I20260620 20:09:34.765206 16223 tablet_bootstrap.cc:492] T f4092ba841454f1f8899ce34a44bfdd6 P 455aeac0af934439bc048191a5560733: No bootstrap required, opened a new log
I20260620 20:09:34.765400 16225 tablet_bootstrap.cc:492] T f4092ba841454f1f8899ce34a44bfdd6 P 597a06fa4de74b659c3e0890b56930c0: No bootstrap required, opened a new log
I20260620 20:09:34.765441 16223 ts_tablet_manager.cc:1397] T f4092ba841454f1f8899ce34a44bfdd6 P 455aeac0af934439bc048191a5560733: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260620 20:09:34.765501 16224 tablet_bootstrap.cc:492] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87: No bootstrap required, opened a new log
I20260620 20:09:34.765564 16225 ts_tablet_manager.cc:1397] T f4092ba841454f1f8899ce34a44bfdd6 P 597a06fa4de74b659c3e0890b56930c0: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260620 20:09:34.765686 16224 ts_tablet_manager.cc:1397] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 20:09:34.768483 16224 raft_consensus.cc:348] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "455aeac0af934439bc048191a5560733" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 38141 } } peers { permanent_uuid: "a0f650c5f0e8419493db248c439a4b87" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45771 } } peers { permanent_uuid: "597a06fa4de74b659c3e0890b56930c0" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44713 } }
I20260620 20:09:34.768705 16224 raft_consensus.cc:374] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:34.768756 16224 raft_consensus.cc:729] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0f650c5f0e8419493db248c439a4b87, State: Initialized, Role: FOLLOWER
I20260620 20:09:34.769172 16224 consensus_queue.cc:260] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "455aeac0af934439bc048191a5560733" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 38141 } } peers { permanent_uuid: "a0f650c5f0e8419493db248c439a4b87" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45771 } } peers { permanent_uuid: "597a06fa4de74b659c3e0890b56930c0" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44713 } }
I20260620 20:09:34.769418 16225 raft_consensus.cc:348] T f4092ba841454f1f8899ce34a44bfdd6 P 597a06fa4de74b659c3e0890b56930c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "455aeac0af934439bc048191a5560733" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 38141 } } peers { permanent_uuid: "a0f650c5f0e8419493db248c439a4b87" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45771 } } peers { permanent_uuid: "597a06fa4de74b659c3e0890b56930c0" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44713 } }
I20260620 20:09:34.769416 16223 raft_consensus.cc:348] T f4092ba841454f1f8899ce34a44bfdd6 P 455aeac0af934439bc048191a5560733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "455aeac0af934439bc048191a5560733" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 38141 } } peers { permanent_uuid: "a0f650c5f0e8419493db248c439a4b87" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45771 } } peers { permanent_uuid: "597a06fa4de74b659c3e0890b56930c0" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44713 } }
I20260620 20:09:34.769670 16223 raft_consensus.cc:374] T f4092ba841454f1f8899ce34a44bfdd6 P 455aeac0af934439bc048191a5560733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:34.769670 16225 raft_consensus.cc:374] T f4092ba841454f1f8899ce34a44bfdd6 P 597a06fa4de74b659c3e0890b56930c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:09:34.769731 16225 raft_consensus.cc:729] T f4092ba841454f1f8899ce34a44bfdd6 P 597a06fa4de74b659c3e0890b56930c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 597a06fa4de74b659c3e0890b56930c0, State: Initialized, Role: FOLLOWER
I20260620 20:09:34.769755 16223 raft_consensus.cc:729] T f4092ba841454f1f8899ce34a44bfdd6 P 455aeac0af934439bc048191a5560733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 455aeac0af934439bc048191a5560733, State: Initialized, Role: FOLLOWER
I20260620 20:09:34.770159 16224 ts_tablet_manager.cc:1428] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260620 20:09:34.770195 16223 consensus_queue.cc:260] T f4092ba841454f1f8899ce34a44bfdd6 P 455aeac0af934439bc048191a5560733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "455aeac0af934439bc048191a5560733" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 38141 } } peers { permanent_uuid: "a0f650c5f0e8419493db248c439a4b87" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45771 } } peers { permanent_uuid: "597a06fa4de74b659c3e0890b56930c0" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44713 } }
I20260620 20:09:34.770196 16225 consensus_queue.cc:260] T f4092ba841454f1f8899ce34a44bfdd6 P 597a06fa4de74b659c3e0890b56930c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "455aeac0af934439bc048191a5560733" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 38141 } } peers { permanent_uuid: "a0f650c5f0e8419493db248c439a4b87" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45771 } } peers { permanent_uuid: "597a06fa4de74b659c3e0890b56930c0" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44713 } }
I20260620 20:09:34.770522 16071 heartbeater.cc:503] Master 127.5.91.190:39251 was elected leader, sending a full tablet report...
I20260620 20:09:34.770783 15936 heartbeater.cc:503] Master 127.5.91.190:39251 was elected leader, sending a full tablet report...
I20260620 20:09:34.771196 16223 ts_tablet_manager.cc:1428] T f4092ba841454f1f8899ce34a44bfdd6 P 455aeac0af934439bc048191a5560733: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260620 20:09:34.771497 16206 heartbeater.cc:503] Master 127.5.91.190:39251 was elected leader, sending a full tablet report...
I20260620 20:09:34.772174 16225 ts_tablet_manager.cc:1428] T f4092ba841454f1f8899ce34a44bfdd6 P 597a06fa4de74b659c3e0890b56930c0: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260620 20:09:34.778683 16229 raft_consensus.cc:482] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:09:34.778837 16229 raft_consensus.cc:504] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "455aeac0af934439bc048191a5560733" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 38141 } } peers { permanent_uuid: "a0f650c5f0e8419493db248c439a4b87" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45771 } } peers { permanent_uuid: "597a06fa4de74b659c3e0890b56930c0" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44713 } }
I20260620 20:09:34.779845 16229 leader_election.cc:288] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 455aeac0af934439bc048191a5560733 (127.5.91.131:38141), 597a06fa4de74b659c3e0890b56930c0 (127.5.91.129:44713)
I20260620 20:09:34.783524 16218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:34.780163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:50761 (local address 127.5.91.131:38141)
0620 20:09:34.780282 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:34.780286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:34.780537 (+ 251us) server_negotiation.cc:407] Connection header received
0620 20:09:34.780651 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:34.780654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:34.780701 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:34.780767 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:34.781060 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.781564 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.782655 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.783038 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.783134 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:34.783184 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:34.783345 (+ 161us) server_negotiation.cc:299] Negotiation successful
0620 20:09:34.783408 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 20:09:34.783545 16232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:34.780098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:38141 (local address 127.5.91.130:50761)
0620 20:09:34.780444 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:34.780457 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:34.780568 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:34.780779 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:34.780782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:34.780799 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:34.780938 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:34.780946 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.781659 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.781662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:34.782552 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:34.782561 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.783134 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.783137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:34.783242 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:34.783260 (+ 18us) client_negotiation.cc:769] Sending connection context
0620 20:09:34.783324 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 20:09:34.783393 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":29,"thread_start_us":100,"threads_started":1}
I20260620 20:09:34.783550 16222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:34.780131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:52427 (local address 127.5.91.129:44713)
0620 20:09:34.780250 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:34.780254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:34.780331 (+ 77us) server_negotiation.cc:407] Connection header received
0620 20:09:34.780491 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:34.780496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:34.780546 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:34.780613 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:09:34.780994 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.781526 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.782664 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.783042 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.783140 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:34.783188 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:34.783367 (+ 179us) server_negotiation.cc:299] Negotiation successful
0620 20:09:34.783443 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 20:09:34.783550 16073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:34.780075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:44713 (local address 127.5.91.130:52427)
0620 20:09:34.780247 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:34.780261 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:09:34.780364 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:09:34.780720 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:09:34.780723 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:09:34.780740 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:09:34.780871 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:34.780878 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.781624 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.781627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:34.782556 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:09:34.782565 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.783140 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.783143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:09:34.783265 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:34.783281 (+ 16us) client_negotiation.cc:769] Sending connection context
0620 20:09:34.783344 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 20:09:34.783418 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":88}
I20260620 20:09:34.784498 16160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4092ba841454f1f8899ce34a44bfdd6" candidate_uuid: "a0f650c5f0e8419493db248c439a4b87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "455aeac0af934439bc048191a5560733" is_pre_election: true
I20260620 20:09:34.784788 16160 raft_consensus.cc:2454] T f4092ba841454f1f8899ce34a44bfdd6 P 455aeac0af934439bc048191a5560733 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0f650c5f0e8419493db248c439a4b87 in term 0.
I20260620 20:09:34.784790 15890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4092ba841454f1f8899ce34a44bfdd6" candidate_uuid: "a0f650c5f0e8419493db248c439a4b87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "597a06fa4de74b659c3e0890b56930c0" is_pre_election: true
I20260620 20:09:34.785063 15890 raft_consensus.cc:2454] T f4092ba841454f1f8899ce34a44bfdd6 P 597a06fa4de74b659c3e0890b56930c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0f650c5f0e8419493db248c439a4b87 in term 0.
I20260620 20:09:34.785207 15957 leader_election.cc:302] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 455aeac0af934439bc048191a5560733, a0f650c5f0e8419493db248c439a4b87; no voters:
I20260620 20:09:34.785466 16229 raft_consensus.cc:2781] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:09:34.785550 16229 raft_consensus.cc:482] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:09:34.785591 16229 raft_consensus.cc:3037] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:34.786590 16229 raft_consensus.cc:504] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "455aeac0af934439bc048191a5560733" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 38141 } } peers { permanent_uuid: "a0f650c5f0e8419493db248c439a4b87" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45771 } } peers { permanent_uuid: "597a06fa4de74b659c3e0890b56930c0" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44713 } }
I20260620 20:09:34.786913 16229 leader_election.cc:288] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87 [CANDIDATE]: Term 1 election: Requested vote from peers 455aeac0af934439bc048191a5560733 (127.5.91.131:38141), 597a06fa4de74b659c3e0890b56930c0 (127.5.91.129:44713)
I20260620 20:09:34.787259 15890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4092ba841454f1f8899ce34a44bfdd6" candidate_uuid: "a0f650c5f0e8419493db248c439a4b87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "597a06fa4de74b659c3e0890b56930c0"
I20260620 20:09:34.787256 16160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4092ba841454f1f8899ce34a44bfdd6" candidate_uuid: "a0f650c5f0e8419493db248c439a4b87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "455aeac0af934439bc048191a5560733"
I20260620 20:09:34.787384 16160 raft_consensus.cc:3037] T f4092ba841454f1f8899ce34a44bfdd6 P 455aeac0af934439bc048191a5560733 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:34.787385 15890 raft_consensus.cc:3037] T f4092ba841454f1f8899ce34a44bfdd6 P 597a06fa4de74b659c3e0890b56930c0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:09:34.788579 16160 raft_consensus.cc:2454] T f4092ba841454f1f8899ce34a44bfdd6 P 455aeac0af934439bc048191a5560733 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0f650c5f0e8419493db248c439a4b87 in term 1.
I20260620 20:09:34.788579 15890 raft_consensus.cc:2454] T f4092ba841454f1f8899ce34a44bfdd6 P 597a06fa4de74b659c3e0890b56930c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0f650c5f0e8419493db248c439a4b87 in term 1.
I20260620 20:09:34.788866 15959 leader_election.cc:302] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597a06fa4de74b659c3e0890b56930c0, a0f650c5f0e8419493db248c439a4b87; no voters:
I20260620 20:09:34.789065 16229 raft_consensus.cc:2781] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:09:34.789291 16229 raft_consensus.cc:686] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87 [term 1 LEADER]: Becoming Leader. State: Replica: a0f650c5f0e8419493db248c439a4b87, State: Running, Role: LEADER
I20260620 20:09:34.789642 16229 consensus_queue.cc:237] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "455aeac0af934439bc048191a5560733" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 38141 } } peers { permanent_uuid: "a0f650c5f0e8419493db248c439a4b87" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45771 } } peers { permanent_uuid: "597a06fa4de74b659c3e0890b56930c0" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44713 } }
I20260620 20:09:34.792943 15742 catalog_manager.cc:5495] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0f650c5f0e8419493db248c439a4b87 (127.5.91.130). New cstate: current_term: 1 leader_uuid: "a0f650c5f0e8419493db248c439a4b87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "455aeac0af934439bc048191a5560733" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 38141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0f650c5f0e8419493db248c439a4b87" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597a06fa4de74b659c3e0890b56930c0" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44713 } health_report { overall_health: UNKNOWN } } }
I20260620 20:09:34.811920 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:34.808611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60912 (local address 127.5.91.130:45771)
0620 20:09:34.808725 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 20:09:34.808728 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:09:34.808740 (+ 12us) server_negotiation.cc:407] Connection header received
0620 20:09:34.808809 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:09:34.808811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:09:34.808862 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:09:34.808996 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:09:34.809136 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.809619 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.810524 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:09:34.810750 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:09:34.810874 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:09:34.811152 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:09:34.811547 (+ 395us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:09:34.811629 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 20:09:34.811708 (+ 79us) server_negotiation.cc:299] Negotiation successful
0620 20:09:34.811758 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 20:09:34.818084 16160 raft_consensus.cc:1261] T f4092ba841454f1f8899ce34a44bfdd6 P 455aeac0af934439bc048191a5560733 [term 1 FOLLOWER]: Refusing update from remote peer a0f650c5f0e8419493db248c439a4b87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:09:34.818094 15890 raft_consensus.cc:1261] T f4092ba841454f1f8899ce34a44bfdd6 P 597a06fa4de74b659c3e0890b56930c0 [term 1 FOLLOWER]: Refusing update from remote peer a0f650c5f0e8419493db248c439a4b87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:09:34.818755 16233 consensus_queue.cc:1035] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "455aeac0af934439bc048191a5560733" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:09:34.818966 16229 consensus_queue.cc:1035] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597a06fa4de74b659c3e0890b56930c0" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:09:34.826839 16237 mvcc.cc:205] Tried to move back new op lower bound from 7299015372047994880 to 7299015371941257216. Current Snapshot: MvccSnapshot[applied={T|T < 7299015372047994880}]
I20260620 20:09:34.835453 15740 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:09:34.835697 15740 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:09:34.837970 15740 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8cf6c695975642aba23c08088796c203: Sending DeleteTablet for 3 replicas of tablet f4092ba841454f1f8899ce34a44bfdd6
I20260620 20:09:34.838760 16140 tablet_service.cc:1514] Processing DeleteTablet for tablet f4092ba841454f1f8899ce34a44bfdd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:09:34 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:36890
I20260620 20:09:34.838888 16005 tablet_service.cc:1514] Processing DeleteTablet for tablet f4092ba841454f1f8899ce34a44bfdd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:09:34 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:60898
I20260620 20:09:34.839270 16246 tablet_replica.cc:330] T f4092ba841454f1f8899ce34a44bfdd6 P 455aeac0af934439bc048191a5560733: stopping tablet replica
I20260620 20:09:34.839272 15870 tablet_service.cc:1514] Processing DeleteTablet for tablet f4092ba841454f1f8899ce34a44bfdd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:09:34 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:46020
I20260620 20:09:34.839576 16246 raft_consensus.cc:2229] T f4092ba841454f1f8899ce34a44bfdd6 P 455aeac0af934439bc048191a5560733 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:09:34.839643 16248 tablet_replica.cc:330] T f4092ba841454f1f8899ce34a44bfdd6 P 597a06fa4de74b659c3e0890b56930c0: stopping tablet replica
I20260620 20:09:34.839898 16246 raft_consensus.cc:2258] T f4092ba841454f1f8899ce34a44bfdd6 P 455aeac0af934439bc048191a5560733 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:09:34.840201 16247 tablet_replica.cc:330] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87: stopping tablet replica
I20260620 20:09:34.840212 16248 raft_consensus.cc:2229] T f4092ba841454f1f8899ce34a44bfdd6 P 597a06fa4de74b659c3e0890b56930c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:09:34.840369 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 15806
I20260620 20:09:34.841429 16247 raft_consensus.cc:2229] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87 [term 1 LEADER]: Raft consensus shutting down.
I20260620 20:09:34.841730 16247 raft_consensus.cc:2258] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:09:34.843267 16247 ts_tablet_manager.cc:1905] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:09:34.846704 16247 ts_tablet_manager.cc:1918] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:09:34.846803 16247 log.cc:1196] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-1/wal/wals/f4092ba841454f1f8899ce34a44bfdd6
I20260620 20:09:34.847096 16247 ts_tablet_manager.cc:1939] T f4092ba841454f1f8899ce34a44bfdd6 P a0f650c5f0e8419493db248c439a4b87: Deleting consensus metadata
I20260620 20:09:34.847697 16246 ts_tablet_manager.cc:1905] T f4092ba841454f1f8899ce34a44bfdd6 P 455aeac0af934439bc048191a5560733: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:09:34.848361 15729 catalog_manager.cc:4841] TS a0f650c5f0e8419493db248c439a4b87 (127.5.91.130:45771): tablet f4092ba841454f1f8899ce34a44bfdd6 (table test-table [id=70ab770bd4d542f983390e4cccbd50ff]) successfully deleted
I20260620 20:09:34.848649 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 15941
W20260620 20:09:34.848649 15729 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.91.129:44713 (error 108)
I20260620 20:09:34.851643 16221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:09:34.851312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:44713 (local address 127.0.0.1:46026)
0620 20:09:34.851471 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 20:09:34.851544 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.129:44713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260620 20:09:34.851895 15729 catalog_manager.cc:4584] TS 597a06fa4de74b659c3e0890b56930c0 (127.5.91.129:44713): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f4092ba841454f1f8899ce34a44bfdd6: Network error: Client connection negotiation failed: client connection to 127.5.91.129:44713: connect: Connection refused (error 111)
I20260620 20:09:34.852408 16246 ts_tablet_manager.cc:1918] T f4092ba841454f1f8899ce34a44bfdd6 P 455aeac0af934439bc048191a5560733: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:09:34.852499 16246 log.cc:1196] T f4092ba841454f1f8899ce34a44bfdd6 P 455aeac0af934439bc048191a5560733: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/f4092ba841454f1f8899ce34a44bfdd6
I20260620 20:09:34.852744 16246 ts_tablet_manager.cc:1939] T f4092ba841454f1f8899ce34a44bfdd6 P 455aeac0af934439bc048191a5560733: Deleting consensus metadata
I20260620 20:09:34.853820 15727 catalog_manager.cc:4841] TS 455aeac0af934439bc048191a5560733 (127.5.91.131:38141): tablet f4092ba841454f1f8899ce34a44bfdd6 (table test-table [id=70ab770bd4d542f983390e4cccbd50ff]) successfully deleted
I20260620 20:09:34.854542 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 16076
I20260620 20:09:34.859894 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 15710
2026-06-20T20:09:34Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2168 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:09:34 dist-test-slave-l8h4 krb5kdc[16252](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:09:34 dist-test-slave-l8h4 krb5kdc[16252](info): set up 2 sockets
Jun 20 20:09:34 dist-test-slave-l8h4 krb5kdc[16252](info): commencing operation
krb5kdc: starting...
W20260620 20:09:35.843804 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.963s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:09:35 dist-test-slave-l8h4 krb5kdc[16252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:09:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:09:35Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.91.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.91.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.91.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/rangeradmin_127.5.91.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.91.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/rangeradmin_127.5.91.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.91.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.91.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.91.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/rangerlookup_127.5.91.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.91.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/rangerlookup_127.5.91.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.91.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.91.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.91.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/HTTP_127.5.91.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.91.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/HTTP_127.5.91.148@KRBTEST.COM.keytab.
I20260620 20:09:36.014734 5486 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskSTSRqO/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/postgres -l logfile start
2026-06-20 20:09:38.883 UTC [16282] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-20 20:09:38.883 UTC [16282] LOG: listening on IPv4 address "127.5.91.148", port 38287
2026-06-20 20:09:38.887 UTC [16282] LOG: listening on Unix socket "/tmp/.s.PGSQL.38287"
2026-06-20 20:09:38.909 UTC [16285] LOG: database system was shut down at 2026-06-20 20:09:37 UTC
2026-06-20 20:09:38.912 UTC [16282] LOG: database system is ready to accept connections
I20260620 20:09:39.807513 5486 mini_postgres.cc:96] Postgres bound to 38287
2026-06-20 20:09:39.813 UTC [16293] FATAL: database "slave" does not exist
127.5.91.148:38287 - accepting connections
I20260620 20:09:39.814087 5486 mini_ranger.cc:161] Starting Ranger...
I20260620 20:09:39.828428 5486 mini_ranger.cc:84] Created miniranger Postgres user
I20260620 20:09:40.440961 5486 mini_ranger.cc:87] Created ranger Postgres database
I20260620 20:09:40.441052 5486 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ranger-admin
2026-06-20 20:09:40,732 [I] DB FLAVOR :POSTGRES
2026-06-20 20:09:40,733 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 20:09:40,733 [I] Checking connection..
2026-06-20 20:09:40,733 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:38287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-20 20:09:40,985 [I] Checking connection passed.
2026-06-20 20:09:40,985 [I] --------- Verifying version history table ---------
2026-06-20 20:09:40,985 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:38287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 20:09:41,249 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 20:09:41,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:38287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 20:09:41,505 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 20:09:41,506 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-20 20:09:41,506 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:38287/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-20 20:09:41.747 UTC [16358] WARNING: there is no transaction in progress
2026-06-20 20:09:41,753 [I] create_dbversion_catalog.sql file imported successfully
2026-06-20 20:09:41,754 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:38287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 20:09:41,993 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-20 20:09:41,994 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 20:09:41,994 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:38287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-20 20:09:42,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:38287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-20 20:09:42,488 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:38287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-l8h4.c.gcp-upstream.internal','N') ;"
2026-06-20 20:09:42,731 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:38287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 20:09:43,003 [I] Table x_portal_user does not exist in database ranger
2026-06-20 20:09:43,003 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:38287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 20:09:43,262 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-20 20:09:43,262 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:38287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 20:09:43,516 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:38287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 20:09:43,773 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-20 20:09:43,773 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:38287/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-20 20:09:44.830 UTC [16466] WARNING: there is no transaction in progress
2026-06-20 20:09:44.849 UTC [16466] WARNING: there is no transaction in progress
2026-06-20 20:09:44.867 UTC [16466] WARNING: there is no transaction in progress
2026-06-20 20:09:44.884 UTC [16466] WARNING: there is no transaction in progress
2026-06-20 20:09:44.901 UTC [16466] WARNING: there is no transaction in progress
2026-06-20 20:09:44.920 UTC [16466] WARNING: there is no transaction in progress
2026-06-20 20:09:44.982 UTC [16466] WARNING: there is no transaction in progress
2026-06-20 20:09:44.991 UTC [16466] WARNING: there is no transaction in progress
2026-06-20 20:09:45.009 UTC [16466] WARNING: there is no transaction in progress
2026-06-20 20:09:45.024 UTC [16466] WARNING: there is no transaction in progress
2026-06-20 20:09:45.036 UTC [16466] WARNING: there is no transaction in progress
2026-06-20 20:09:45.053 UTC [16466] WARNING: there is no transaction in progress
2026-06-20 20:09:45.064 UTC [16466] WARNING: there is no transaction in progress
2026-06-20 20:09:45.072 UTC [16466] WARNING: there is no transaction in progress
2026-06-20 20:09:45.082 UTC [16466] WARNING: there is no transaction in progress
2026-06-20 20:09:45.091 UTC [16466] WARNING: there is no transaction in progress
2026-06-20 20:09:45,875 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-20 20:09:45,875 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:38287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-20 20:09:46,123 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-20 20:09:46,124 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:38287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 20:09:46,359 [I] Table x_portal_user already exists in database 'ranger'
2026-06-20 20:09:46,359 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:38287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 20:09:46,620 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-20 20:09:46,620 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:38287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 20:09:46,885 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:38287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 20:09:47,126 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:38287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-l8h4.c.gcp-upstream.internal';"
2026-06-20 20:09:47,378 [I] CORE_DB_SCHEMA import status has been updated
2026-06-20 20:09:47,378 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:38287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-20 20:09:47,635 [I] DB_PATCHES have already been applied
I20260620 20:09:47.641809 5486 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ranger-admin:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskSTSRqO/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260620 20:09:47.641952 5486 mini_ranger.cc:193] Using host: 127.5.91.148
I20260620 20:09:47.645414 5486 mini_ranger.cc:227] Ranger admin URL: http://127.5.91.148:51189
Jun 20, 2026 8:09:47 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 8:09:47 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 8:09:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ranger-admin/ews/webapp
Jun 20, 2026 8:09:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 20, 2026 8:09:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 20, 2026 8:09:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ranger-admin/ews/webapp].
Jun 20, 2026 8:09:48 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260620 20:09:48.672245 5486 mini_ranger.cc:160] Time spent starting Ranger: real 8.858s user 0.000s sys 0.010s
Jun 20, 2026 8:10:10 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 20, 2026 8:10:10 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 20, 2026 8:10:10 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 20, 2026 8:10:10 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 20, 2026 8:10:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 20, 2026 8:10:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 20, 2026 8:10:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 20, 2026 8:10:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 20, 2026 8:10:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 20, 2026 8:10:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 20, 2026 8:10:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 20, 2026 8:10:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 20, 2026 8:10:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 20, 2026 8:10:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 20, 2026 8:10:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 20, 2026 8:10:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 20, 2026 8:10:10 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 20, 2026 8:10:11 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260620 20:10:12.443029 5486 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.91.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.91.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.91.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/rangerkms_127.5.91.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.91.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/rangerkms_127.5.91.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.91.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.91.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.91.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/HTTP_127.5.91.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.91.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/HTTP_127.5.91.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 20 20:10:12 dist-test-slave-l8h4 krb5kdc[16252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986212, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260620 20:10:12.537514 5486 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260620 20:10:12.548244 5486 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260620 20:10:13.789079 5486 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260620 20:10:13.789171 5486 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ranger-kms
I20260620 20:10:13.789259 5486 mini_ranger_kms.cc:227] Using postgres at 127.5.91.148:38287
2026-06-20 20:10:13,975 [I] DB FLAVOR :POSTGRES
2026-06-20 20:10:13,975 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 20:10:13,975 [I] Checking connection
2026-06-20 20:10:13,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:38287/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-20 20:10:14,242 [I] connection success
2026-06-20 20:10:14,242 [I] --------- Verifying Ranger DB tables ---------
2026-06-20 20:10:14,242 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-20 20:10:14,242 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:38287/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-20 20:10:14,520 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-20 20:10:14,521 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 20:10:14,521 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-20 20:10:14,521 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:38287/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-20 20:10:14,827 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260620 20:10:15.474817 5486 mini_ranger_kms.cc:310] Created kms service
I20260620 20:10:15.700263 5486 mini_ranger_kms.cc:326] Created kudu user
I20260620 20:10:15.755837 5486 mini_ranger_kms.cc:343] Created rangerkms user
Jun 20 20:10:15 dist-test-slave-l8h4 krb5kdc[16252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986212, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.91.148@KRBTEST.COM
I20260620 20:10:16.240909 5486 mini_ranger_kms.cc:384] Added ranger policy
I20260620 20:10:16.241067 5486 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ranger-kms:/tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260620 20:10:16.241087 5486 mini_ranger_kms.cc:240] Using host: 127.5.91.148
I20260620 20:10:16.245890 5486 mini_ranger_kms.cc:276] Ranger KMS PID: 16648
I20260620 20:10:16.246068 5486 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.91.148:43119
Jun 20, 2026 8:10:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 8:10:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 8:10:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 20, 2026 8:10:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 20, 2026 8:10:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ranger-kms/ews/webapp].
Jun 20, 2026 8:10:17 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260620 20:10:18.425622 5486 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.888s user 0.004s sys 0.164s
I20260620 20:10:18.425733 5486 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260620 20:10:18.425848 5486 mini_ranger_kms.cc:401] 127.5.91.148:43119/kms/v1/keys
Jun 20 20:10:18 dist-test-slave-l8h4 krb5kdc[16252](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781986218, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.91.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 20 20:10:19 dist-test-slave-l8h4 krb5kdc[16252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986218, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.91.148@KRBTEST.COM for HTTP/127.5.91.148@KRBTEST.COM
Jun 20 20:10:19 dist-test-slave-l8h4 krb5kdc[16252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986218, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.91.148@KRBTEST.COM for HTTP/127.5.91.148@KRBTEST.COM
Jun 20 20:10:21 dist-test-slave-l8h4 krb5kdc[16252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:10:22.023617 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:32991
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:33511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:32991
--ranger_config_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.91.148:43119/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:10:22.144634 16699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:10:22.144940 16699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:10:22.144990 16699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:10:22.149883 16699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:10:22.149971 16699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:10:22.149991 16699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:10:22.150007 16699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:10:22.150022 16699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:10:22.155958 16699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:33511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.91.148:43119/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:32991
--ranger_config_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:32991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:10:22.157037 16699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:10:22.158016 16699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 20:10:22.166754 16699 server_base.cc:1034] running on GCE node
W20260620 20:10:22.166851 16704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:10:22.166875 16705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:10:22.166980 16707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:10:22.167598 16699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:10:22.168705 16699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:10:22.169884 16699 hybrid_clock.cc:648] HybridClock initialized: now 1781986222169865 us; error 27 us; skew 500 ppm
Jun 20 20:10:22 dist-test-slave-l8h4 krb5kdc[16252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986222, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:10:22.172979 16699 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:10:22.173887 16699 webserver.cc:460] Webserver started at http://127.5.91.190:33729/ using document root <none> and password file <none>
I20260620 20:10:22.174440 16699 fs_manager.cc:359] Metadata directory not provided
I20260620 20:10:22.174496 16699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:10:22.174688 16699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 20:10:22 dist-test-slave-l8h4 krb5kdc[16252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986222, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for HTTP/127.5.91.148@KRBTEST.COM
I20260620 20:10:22.235066 16699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "353940bd102749c284251bc8044eb29c"
format_stamp: "Formatted at 2026-06-20 20:10:22 on dist-test-slave-l8h4"
server_key: "86433ed4dda299de814114893534850c"
server_key_iv: "be9f53e57201ed75a83e1a3fc8e8dd2e"
server_key_version: "kuduclusterkey@0"
I20260620 20:10:22.235684 16699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "353940bd102749c284251bc8044eb29c"
format_stamp: "Formatted at 2026-06-20 20:10:22 on dist-test-slave-l8h4"
server_key: "86433ed4dda299de814114893534850c"
server_key_iv: "be9f53e57201ed75a83e1a3fc8e8dd2e"
server_key_version: "kuduclusterkey@0"
I20260620 20:10:22.239420 16699 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 20:10:22.255777 16714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:10:22.257820 16699 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260620 20:10:22.257995 16699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "353940bd102749c284251bc8044eb29c"
format_stamp: "Formatted at 2026-06-20 20:10:22 on dist-test-slave-l8h4"
server_key: "86433ed4dda299de814114893534850c"
server_key_iv: "be9f53e57201ed75a83e1a3fc8e8dd2e"
server_key_version: "kuduclusterkey@0"
I20260620 20:10:22.258190 16699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:10:22.276743 16699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:10:22.280284 16699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:10:22.280479 16699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:10:22.291008 16699 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:32991
I20260620 20:10:22.291024 16766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:32991 every 8 connection(s)
I20260620 20:10:22.291998 16699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:10:22.292474 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 16699
I20260620 20:10:22.292587 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:10:22.292861 5486 external_mini_cluster.cc:1428] Setting key ac6914fef788b3f4ab6b3ea31f1eaf26
I20260620 20:10:22.295553 16767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:10:22.303051 16767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c: Bootstrap starting.
Jun 20 20:10:22 dist-test-slave-l8h4 krb5kdc[16252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:10:22.305789 16767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c: Neither blocks nor log segments found. Creating new log.
I20260620 20:10:22.306759 16767 log.cc:824] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c: Log is configured to *not* fsync() on all Append() calls
I20260620 20:10:22.309778 16767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c: No bootstrap required, opened a new log
I20260620 20:10:22.311692 16770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:10:22.294380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50088 (local address 127.5.91.190:32991)
0620 20:10:22.294857 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0620 20:10:22.294876 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0620 20:10:22.294919 (+ 43us) server_negotiation.cc:407] Connection header received
0620 20:10:22.295836 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:10:22.295860 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:10:22.296375 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:10:22.296806 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:10:22.298261 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:10:22.299770 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:10:22.301276 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:10:22.302085 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:10:22.302314 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:10:22.305026 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:10:22.305034 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:10:22.305050 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:10:22.305086 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:10:22.307742 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:10:22.308857 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:10:22.308865 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:10:22.308871 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:10:22.309031 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:10:22.309395 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:10:22.309398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:10:22.309400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:10:22.309915 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:10:22.310187 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0620 20:10:22.310468 (+ 281us) server_negotiation.cc:299] Negotiation successful
0620 20:10:22.310754 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":142,"threads_started":1}
I20260620 20:10:22.313412 16767 raft_consensus.cc:348] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "353940bd102749c284251bc8044eb29c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 32991 } }
I20260620 20:10:22.313902 16767 raft_consensus.cc:374] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:10:22.314020 16767 raft_consensus.cc:729] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353940bd102749c284251bc8044eb29c, State: Initialized, Role: FOLLOWER
I20260620 20:10:22.314850 16767 consensus_queue.cc:260] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "353940bd102749c284251bc8044eb29c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 32991 } }
I20260620 20:10:22.315049 16767 raft_consensus.cc:388] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:10:22.315092 16767 raft_consensus.cc:482] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:10:22.315163 16767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:10:22.316669 16767 raft_consensus.cc:504] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "353940bd102749c284251bc8044eb29c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 32991 } }
I20260620 20:10:22.317166 16767 leader_election.cc:302] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 353940bd102749c284251bc8044eb29c; no voters:
I20260620 20:10:22.317550 16767 leader_election.cc:288] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:10:22.317735 16772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:10:22.318025 16772 raft_consensus.cc:686] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c [term 1 LEADER]: Becoming Leader. State: Replica: 353940bd102749c284251bc8044eb29c, State: Running, Role: LEADER
I20260620 20:10:22.318583 16772 consensus_queue.cc:237] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "353940bd102749c284251bc8044eb29c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 32991 } }
I20260620 20:10:22.319161 16767 sys_catalog.cc:564] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:10:22.320463 16773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "353940bd102749c284251bc8044eb29c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "353940bd102749c284251bc8044eb29c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 32991 } } }
I20260620 20:10:22.320458 16774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 353940bd102749c284251bc8044eb29c. Latest consensus state: current_term: 1 leader_uuid: "353940bd102749c284251bc8044eb29c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "353940bd102749c284251bc8044eb29c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 32991 } } }
I20260620 20:10:22.320613 16773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c [sys.catalog]: This master's current role is: LEADER
I20260620 20:10:22.320613 16774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c [sys.catalog]: This master's current role is: LEADER
I20260620 20:10:22.321142 16781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:10:22.323832 16781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:10:22.324275 16767 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 20:10:22.334946 16781 catalog_manager.cc:1347] Generated new cluster ID: 32a35523720b48f4a62ebf118b14410e
I20260620 20:10:22.335029 16781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:10:22.354940 16781 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:10:22.356288 16781 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:10:22.367559 16781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c: Generated new TSK 0
I20260620 20:10:22.368445 16781 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.91.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
I20260620 20:10:23.147171 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.91.148:43119/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:32991
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:33511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:10:23.281800 16812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:10:23.282073 16812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:10:23.282131 16812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:10:23.287127 16812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:10:23.287199 16812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:10:23.287293 16812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:10:23.294296 16812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:33511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.91.148:43119/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:32991
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:10:23.295605 16812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:10:23.296867 16812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:10:23.311280 16820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:10:23.311703 16812 server_base.cc:1034] running on GCE node
W20260620 20:10:23.311973 16817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:10:23.312248 16818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:10:23.312963 16812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:10:23.314038 16812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:10:23.315320 16812 hybrid_clock.cc:648] HybridClock initialized: now 1781986223315288 us; error 55 us; skew 500 ppm
Jun 20 20:10:23 dist-test-slave-l8h4 krb5kdc[16252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986223, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:10:23.320151 16812 init.cc:377] Logged in from keytab as kudu/127.5.91.129@KRBTEST.COM (short username kudu)
I20260620 20:10:23.321372 16812 webserver.cc:460] Webserver started at http://127.5.91.129:40405/ using document root <none> and password file <none>
I20260620 20:10:23.322211 16812 fs_manager.cc:359] Metadata directory not provided
I20260620 20:10:23.322301 16812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:10:23.322553 16812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 20:10:23 dist-test-slave-l8h4 krb5kdc[16252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986223, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for HTTP/127.5.91.148@KRBTEST.COM
I20260620 20:10:23.338335 16812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "5c410fa48d23428693b41b84b7d70807"
format_stamp: "Formatted at 2026-06-20 20:10:23 on dist-test-slave-l8h4"
server_key: "9f33242229c96d17290e0be11bf1ce71"
server_key_iv: "6682830b76594abc7d3d727d4b55518c"
server_key_version: "kuduclusterkey@0"
I20260620 20:10:23.338949 16812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "5c410fa48d23428693b41b84b7d70807"
format_stamp: "Formatted at 2026-06-20 20:10:23 on dist-test-slave-l8h4"
server_key: "9f33242229c96d17290e0be11bf1ce71"
server_key_iv: "6682830b76594abc7d3d727d4b55518c"
server_key_version: "kuduclusterkey@0"
I20260620 20:10:23.343390 16812 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 20:10:23.361596 16830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:10:23.363483 16812 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260620 20:10:23.363622 16812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "5c410fa48d23428693b41b84b7d70807"
format_stamp: "Formatted at 2026-06-20 20:10:23 on dist-test-slave-l8h4"
server_key: "9f33242229c96d17290e0be11bf1ce71"
server_key_iv: "6682830b76594abc7d3d727d4b55518c"
server_key_version: "kuduclusterkey@0"
I20260620 20:10:23.363751 16812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:10:23.391629 16812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:10:23.394491 16812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:10:23.394652 16812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:10:23.395195 16812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:10:23.396142 16812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:10:23.396204 16812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:10:23.396256 16812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:10:23.396284 16812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:10:23.408768 16812 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:41631
I20260620 20:10:23.409982 16812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:10:23.416041 16943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:41631 every 8 connection(s)
I20260620 20:10:23.419376 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 16812
I20260620 20:10:23.419515 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:10:23.419950 5486 external_mini_cluster.cc:1428] Setting key b5190e0803e3473d032421cb31dbe45b
Jun 20 20:10:23 dist-test-slave-l8h4 krb5kdc[16252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986223, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.91.130@KRBTEST.COM; defaulting to no policy
I20260620 20:10:23.438937 16946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:10:23.416474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:46487 (local address 127.5.91.190:32991)
0620 20:10:23.426413 (+ 9939us) server_negotiation.cc:206] Beginning negotiation
0620 20:10:23.426418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:10:23.426439 (+ 21us) server_negotiation.cc:407] Connection header received
0620 20:10:23.426499 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:10:23.426502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:10:23.426557 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:10:23.426669 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:10:23.428478 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:10:23.429363 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:10:23.431127 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:10:23.431496 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:10:23.431619 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:10:23.434488 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:10:23.434492 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:10:23.434495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:10:23.434526 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:10:23.436430 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:10:23.437093 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:10:23.437097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:10:23.437099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:10:23.437190 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:10:23.437588 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:10:23.437591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:10:23.437592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:10:23.437822 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:10:23.437923 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 20:10:23.438555 (+ 632us) server_negotiation.cc:299] Negotiation successful
0620 20:10:23.438741 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9837,"thread_start_us":104,"threads_started":1}
I20260620 20:10:23.439831 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:10:23.416978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:32991 (local address 127.5.91.129:46487)
0620 20:10:23.417460 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0620 20:10:23.417514 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 20:10:23.419055 (+ 1541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:10:23.427106 (+ 8051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:10:23.427119 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:10:23.427521 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:10:23.428285 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:10:23.428306 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:10:23.429519 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:10:23.429522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:10:23.430997 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:10:23.431009 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:10:23.431695 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:10:23.431699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:10:23.431897 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:10:23.432626 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:10:23.432646 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:10:23.434263 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:10:23.436574 (+ 2311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:10:23.436581 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:10:23.436598 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:10:23.436965 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:10:23.437278 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:10:23.437281 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:10:23.437283 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:10:23.437472 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:10:23.437968 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:10:23.437973 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:10:23.438287 (+ 314us) client_negotiation.cc:769] Sending connection context
0620 20:10:23.438480 (+ 193us) client_negotiation.cc:240] Negotiation successful
0620 20:10:23.438836 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":123,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.130@KRBTEST.COM" created.
I20260620 20:10:23.441462 16944 heartbeater.cc:347] Connected to a master server at 127.5.91.190:32991
I20260620 20:10:23.441715 16944 heartbeater.cc:464] Registering TS with master...
I20260620 20:10:23.442312 16944 heartbeater.cc:511] Master 127.5.91.190:32991 requested a full tablet report, sending...
I20260620 20:10:23.443987 16731 ts_manager.cc:194] Registered new tserver with Master: 5c410fa48d23428693b41b84b7d70807 (127.5.91.129:41631)
I20260620 20:10:23.445544 16731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:46487
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
I20260620 20:10:23.462944 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.91.148:43119/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:32991
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:33511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 20:10:23 dist-test-slave-l8h4 krb5kdc[16252](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781986223, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260620 20:10:23.632881 16952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:10:23.633277 16952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:10:23.633358 16952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:10:23.639847 16952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:10:23.639952 16952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:10:23.640045 16952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:10:23.648288 16952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:33511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.91.148:43119/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:32991
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:10:23.649749 16952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:10:23.651153 16952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:10:23.664036 16961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:10:23.664496 16959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:10:23.664011 16958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:10:23.665028 16952 server_base.cc:1034] running on GCE node
I20260620 20:10:23.665598 16952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:10:23.666497 16952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:10:23.667775 16952 hybrid_clock.cc:648] HybridClock initialized: now 1781986223667738 us; error 50 us; skew 500 ppm
Jun 20 20:10:23 dist-test-slave-l8h4 krb5kdc[16252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986223, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:10:23.671840 16952 init.cc:377] Logged in from keytab as kudu/127.5.91.130@KRBTEST.COM (short username kudu)
I20260620 20:10:23.673035 16952 webserver.cc:460] Webserver started at http://127.5.91.130:38277/ using document root <none> and password file <none>
I20260620 20:10:23.673628 16952 fs_manager.cc:359] Metadata directory not provided
I20260620 20:10:23.673699 16952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:10:23.673871 16952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 20:10:23 dist-test-slave-l8h4 krb5kdc[16252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986223, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for HTTP/127.5.91.148@KRBTEST.COM
I20260620 20:10:23.691298 16952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "172545da50194bea9c7b4e3103c12138"
format_stamp: "Formatted at 2026-06-20 20:10:23 on dist-test-slave-l8h4"
server_key: "0b33dd6c9e7f8d3c1aae551ff57f8728"
server_key_iv: "4d7312f7ea8c3962d267d0362c2eccef"
server_key_version: "kuduclusterkey@0"
I20260620 20:10:23.691962 16952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "172545da50194bea9c7b4e3103c12138"
format_stamp: "Formatted at 2026-06-20 20:10:23 on dist-test-slave-l8h4"
server_key: "0b33dd6c9e7f8d3c1aae551ff57f8728"
server_key_iv: "4d7312f7ea8c3962d267d0362c2eccef"
server_key_version: "kuduclusterkey@0"
I20260620 20:10:23.695911 16952 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 20:10:23.728434 16968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:10:23.730392 16952 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.000s
I20260620 20:10:23.730549 16952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "172545da50194bea9c7b4e3103c12138"
format_stamp: "Formatted at 2026-06-20 20:10:23 on dist-test-slave-l8h4"
server_key: "0b33dd6c9e7f8d3c1aae551ff57f8728"
server_key_iv: "4d7312f7ea8c3962d267d0362c2eccef"
server_key_version: "kuduclusterkey@0"
I20260620 20:10:23.730695 16952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:10:23.759963 16952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:10:23.763931 16952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:10:23.764168 16952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:10:23.765012 16952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:10:23.766220 16952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:10:23.766304 16952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:10:23.766371 16952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:10:23.766407 16952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:10:23.788412 16952 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:35989
I20260620 20:10:23.789633 16952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:10:23.794720 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 16952
I20260620 20:10:23.794852 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:10:23.791144 17082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:35989 every 8 connection(s)
I20260620 20:10:23.795150 5486 external_mini_cluster.cc:1428] Setting key 2119f746b455a71630847f35df55ad02
Jun 20 20:10:23 dist-test-slave-l8h4 krb5kdc[16252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986223, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:10:23.813045 16946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:10:23.794544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:41581 (local address 127.5.91.190:32991)
0620 20:10:23.794691 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0620 20:10:23.794696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:10:23.795560 (+ 864us) server_negotiation.cc:407] Connection header received
0620 20:10:23.797630 (+ 2070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:10:23.797634 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:10:23.797686 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:10:23.797739 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:10:23.799468 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:10:23.800264 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:10:23.802014 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:10:23.802365 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:10:23.802447 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:10:23.805524 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:10:23.805528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:10:23.805531 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:10:23.805559 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:10:23.807320 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:10:23.810982 (+ 3662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:10:23.810987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:10:23.810989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:10:23.811098 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:10:23.811579 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:10:23.811582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:10:23.811584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:10:23.811831 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:10:23.811935 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 20:10:23.812689 (+ 754us) server_negotiation.cc:299] Negotiation successful
0620 20:10:23.812847 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 20:10:23.814808 17085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:10:23.793387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:32991 (local address 127.5.91.130:41581)
0620 20:10:23.795356 (+ 1969us) negotiation.cc:107] Waiting for socket to connect
0620 20:10:23.795413 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0620 20:10:23.797378 (+ 1965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:10:23.798200 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:10:23.798210 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:10:23.798572 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:10:23.799289 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:10:23.799309 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:10:23.800425 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:10:23.800429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:10:23.801852 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:10:23.801864 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:10:23.802665 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:10:23.802670 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:10:23.802883 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:10:23.803592 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:10:23.803615 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:10:23.805316 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:10:23.807475 (+ 2159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:10:23.807481 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:10:23.807500 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:10:23.807866 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:10:23.811228 (+ 3362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:10:23.811231 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:10:23.811233 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:10:23.811460 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:10:23.812010 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:10:23.812018 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:10:23.812388 (+ 370us) client_negotiation.cc:769] Sending connection context
0620 20:10:23.813297 (+ 909us) client_negotiation.cc:240] Negotiation successful
0620 20:10:23.813681 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1746,"thread_start_us":144,"threads_started":1}
I20260620 20:10:23.816991 17083 heartbeater.cc:347] Connected to a master server at 127.5.91.190:32991
I20260620 20:10:23.817255 17083 heartbeater.cc:464] Registering TS with master...
I20260620 20:10:23.817860 17083 heartbeater.cc:511] Master 127.5.91.190:32991 requested a full tablet report, sending...
I20260620 20:10:23.819141 16731 ts_manager.cc:194] Registered new tserver with Master: 172545da50194bea9c7b4e3103c12138 (127.5.91.130:35989)
I20260620 20:10:23.819908 16731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:41581
WARNING: no policy specified for kudu/127.5.91.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
I20260620 20:10:23.846740 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.91.148:43119/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:32991
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:33511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:10:24.011291 17088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:10:24.011752 17088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:10:24.011863 17088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
Jun 20 20:10:24 dist-test-slave-l8h4 krb5kdc[16252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986223, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for HTTP/127.5.91.148@KRBTEST.COM
W20260620 20:10:24.019122 17088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:10:24.019309 17088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:10:24.019435 17088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:10:24.027511 17088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:33511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.91.148:43119/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:32991
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:10:24.029066 17088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:10:24.030889 17088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:10:24.043682 17093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:10:24.043678 17096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:10:24.046705 17094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:10:24.048081 17088 server_base.cc:1034] running on GCE node
I20260620 20:10:24.048590 17088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:10:24.049407 17088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:10:24.050916 17088 hybrid_clock.cc:648] HybridClock initialized: now 1781986224050829 us; error 98 us; skew 500 ppm
Jun 20 20:10:24 dist-test-slave-l8h4 krb5kdc[16252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986224, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:10:24.054873 17088 init.cc:377] Logged in from keytab as kudu/127.5.91.131@KRBTEST.COM (short username kudu)
I20260620 20:10:24.056115 17088 webserver.cc:460] Webserver started at http://127.5.91.131:45419/ using document root <none> and password file <none>
I20260620 20:10:24.056879 17088 fs_manager.cc:359] Metadata directory not provided
I20260620 20:10:24.056989 17088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:10:24.057246 17088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 20:10:24 dist-test-slave-l8h4 krb5kdc[16252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986224, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for HTTP/127.5.91.148@KRBTEST.COM
I20260620 20:10:24.096001 17088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "52a87cfecb9e4a569cb3340b709d580c"
format_stamp: "Formatted at 2026-06-20 20:10:24 on dist-test-slave-l8h4"
server_key: "72f49132b9bdb2e649d3366e95b37b36"
server_key_iv: "44b22f993926e4f4e13e1f1cf2bf92bf"
server_key_version: "kuduclusterkey@0"
I20260620 20:10:24.096736 17088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "52a87cfecb9e4a569cb3340b709d580c"
format_stamp: "Formatted at 2026-06-20 20:10:24 on dist-test-slave-l8h4"
server_key: "72f49132b9bdb2e649d3366e95b37b36"
server_key_iv: "44b22f993926e4f4e13e1f1cf2bf92bf"
server_key_version: "kuduclusterkey@0"
I20260620 20:10:24.101636 17088 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 20:10:24.128516 17103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:10:24.130388 17088 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.003s sys 0.000s
I20260620 20:10:24.130527 17088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "52a87cfecb9e4a569cb3340b709d580c"
format_stamp: "Formatted at 2026-06-20 20:10:24 on dist-test-slave-l8h4"
server_key: "72f49132b9bdb2e649d3366e95b37b36"
server_key_iv: "44b22f993926e4f4e13e1f1cf2bf92bf"
server_key_version: "kuduclusterkey@0"
I20260620 20:10:24.130666 17088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:10:24.148240 17088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:10:24.151746 17088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:10:24.151926 17088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:10:24.152571 17088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:10:24.153522 17088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:10:24.153589 17088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:10:24.153637 17088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:10:24.153664 17088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:10:24.166098 17088 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:40601
I20260620 20:10:24.167260 17088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:10:24.167977 17217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:40601 every 8 connection(s)
I20260620 20:10:24.177440 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 17088
I20260620 20:10:24.177575 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:10:24.177862 5486 external_mini_cluster.cc:1428] Setting key 58debb18939798cc63f91c44bf99511c
Jun 20 20:10:24 dist-test-slave-l8h4 krb5kdc[16252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986224, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:10:24.189713 16946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:10:24.171007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:38777 (local address 127.5.91.190:32991)
0620 20:10:24.171152 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 20:10:24.171157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:10:24.174646 (+ 3489us) server_negotiation.cc:407] Connection header received
0620 20:10:24.175848 (+ 1202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:10:24.175851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:10:24.175905 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:10:24.175997 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:10:24.178985 (+ 2988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.179810 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.181874 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.182237 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.182354 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:10:24.185492 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:10:24.185497 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:10:24.185500 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:10:24.185528 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:10:24.187182 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:10:24.187837 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:10:24.187840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:10:24.187842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:10:24.187926 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:10:24.188383 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:10:24.188386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:10:24.188388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:10:24.188606 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:10:24.188694 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 20:10:24.189423 (+ 729us) server_negotiation.cc:299] Negotiation successful
0620 20:10:24.189553 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 20:10:24.191393 17220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:10:24.169885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:32991 (local address 127.5.91.131:38777)
0620 20:10:24.174468 (+ 4583us) negotiation.cc:107] Waiting for socket to connect
0620 20:10:24.174527 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0620 20:10:24.175620 (+ 1093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:10:24.176183 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:10:24.176192 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:10:24.176548 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:10:24.178819 (+ 2271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:10:24.178843 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.179964 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.179967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:10:24.181685 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:10:24.181697 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.182399 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.182403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:10:24.182603 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:10:24.183384 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:10:24.183405 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:10:24.185284 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:10:24.187319 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:10:24.187326 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:10:24.187355 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:10:24.187731 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:10:24.188082 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:10:24.188085 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:10:24.188087 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:10:24.188291 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:10:24.188714 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:10:24.188722 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:10:24.189122 (+ 400us) client_negotiation.cc:769] Sending connection context
0620 20:10:24.189376 (+ 254us) client_negotiation.cc:240] Negotiation successful
0620 20:10:24.189731 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4345,"thread_start_us":163,"threads_started":1}
I20260620 20:10:24.193213 17218 heartbeater.cc:347] Connected to a master server at 127.5.91.190:32991
I20260620 20:10:24.193480 17218 heartbeater.cc:464] Registering TS with master...
I20260620 20:10:24.194096 17218 heartbeater.cc:511] Master 127.5.91.190:32991 requested a full tablet report, sending...
I20260620 20:10:24.195290 16731 ts_manager.cc:194] Registered new tserver with Master: 52a87cfecb9e4a569cb3340b709d580c (127.5.91.131:40601)
I20260620 20:10:24.196235 16731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:38777
I20260620 20:10:24.203960 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 20:10:24.219704 16946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:10:24.206392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39042 (local address 127.5.91.190:32991)
0620 20:10:24.206509 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 20:10:24.206514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:10:24.210586 (+ 4072us) server_negotiation.cc:407] Connection header received
0620 20:10:24.210703 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:10:24.210706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:10:24.210765 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:10:24.210864 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:10:24.211679 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.212503 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.213647 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.213990 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.214301 (+ 311us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:10:24.215942 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:10:24.215955 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:10:24.215958 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:10:24.215988 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:10:24.217634 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:10:24.218276 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:10:24.218280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:10:24.218282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:10:24.218376 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:10:24.218669 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:10:24.218673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:10:24.218674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:10:24.218908 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:10:24.219050 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0620 20:10:24.219402 (+ 352us) server_negotiation.cc:299] Negotiation successful
0620 20:10:24.219534 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 20:10:24.224203 16731 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:10:24.229108 16731 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:10:24.246687 17230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:10:24.239834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:35989 (local address 127.0.0.1:57854)
0620 20:10:24.240198 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0620 20:10:24.240251 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 20:10:24.240390 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:10:24.241495 (+ 1105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:10:24.241501 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:10:24.241526 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:10:24.241679 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:10:24.241687 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.243383 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.243387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:10:24.244810 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:10:24.244821 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.246071 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.246074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:10:24.246255 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:10:24.246304 (+ 49us) client_negotiation.cc:769] Sending connection context
0620 20:10:24.246395 (+ 91us) client_negotiation.cc:240] Negotiation successful
0620 20:10:24.246531 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":97,"threads_started":1}
I20260620 20:10:24.247871 17231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:10:24.240672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57854 (local address 127.5.91.130:35989)
0620 20:10:24.240975 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0620 20:10:24.240989 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 20:10:24.241012 (+ 23us) server_negotiation.cc:407] Connection header received
0620 20:10:24.241086 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:10:24.241091 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:10:24.241310 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:10:24.241834 (+ 524us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:10:24.241963 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.243237 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.244975 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.245918 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.247374 (+ 1456us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:10:24.247577 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0620 20:10:24.247632 (+ 55us) server_negotiation.cc:299] Negotiation successful
0620 20:10:24.247731 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":110,"threads_started":1}
I20260620 20:10:24.251199 17234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:10:24.241395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:40601 (local address 127.0.0.1:58542)
0620 20:10:24.243494 (+ 2099us) negotiation.cc:107] Waiting for socket to connect
0620 20:10:24.243508 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:10:24.243589 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:10:24.245546 (+ 1957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:10:24.245549 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:10:24.245566 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:10:24.245665 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:10:24.245671 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.247811 (+ 2140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.247815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:10:24.249081 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:10:24.249092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.250776 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.250779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:10:24.250886 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:10:24.250911 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 20:10:24.250975 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 20:10:24.251057 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2035,"thread_start_us":75,"threads_started":1}
I20260620 20:10:24.252228 17233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:10:24.241746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58542 (local address 127.5.91.131:40601)
0620 20:10:24.244654 (+ 2908us) server_negotiation.cc:206] Beginning negotiation
0620 20:10:24.244931 (+ 277us) server_negotiation.cc:399] Waiting for connection header
0620 20:10:24.244958 (+ 27us) server_negotiation.cc:407] Connection header received
0620 20:10:24.245039 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:10:24.245045 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:10:24.245261 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:10:24.245398 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:10:24.245790 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.247684 (+ 1894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.249257 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.250478 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.250576 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:10:24.250712 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0620 20:10:24.251483 (+ 771us) server_negotiation.cc:299] Negotiation successful
0620 20:10:24.251590 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2778,"thread_start_us":122,"threads_started":1}
I20260620 20:10:24.251667 17016 tablet_service.cc:1467] Processing CreateTablet for tablet 947100db3a5c43cebaa04e97c97a503e (DEFAULT_TABLE table=test-table [id=fac1b29efaf54a7aa3169c18311242ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:10:24.253101 17016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 947100db3a5c43cebaa04e97c97a503e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:10:24.255218 17133 tablet_service.cc:1467] Processing CreateTablet for tablet 947100db3a5c43cebaa04e97c97a503e (DEFAULT_TABLE table=test-table [id=fac1b29efaf54a7aa3169c18311242ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:10:24.256403 17133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 947100db3a5c43cebaa04e97c97a503e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:10:24.261282 17232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:10:24.242517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:41631 (local address 127.0.0.1:39078)
0620 20:10:24.243764 (+ 1247us) negotiation.cc:107] Waiting for socket to connect
0620 20:10:24.243775 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 20:10:24.244364 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:10:24.251028 (+ 6664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:10:24.251031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:10:24.251053 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:10:24.251149 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:10:24.251155 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.258351 (+ 7196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.258356 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:10:24.259469 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:10:24.259480 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.260856 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.260860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:10:24.260969 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:10:24.261000 (+ 31us) client_negotiation.cc:769] Sending connection context
0620 20:10:24.261049 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 20:10:24.261130 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1190,"thread_start_us":84,"threads_started":1}
I20260620 20:10:24.262008 17235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:10:24.242670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39078 (local address 127.5.91.129:41631)
0620 20:10:24.250372 (+ 7702us) server_negotiation.cc:206] Beginning negotiation
0620 20:10:24.250380 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 20:10:24.250406 (+ 26us) server_negotiation.cc:407] Connection header received
0620 20:10:24.250481 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:10:24.250486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:10:24.250689 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:10:24.250853 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:10:24.251279 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.252546 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.259641 (+ 7095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.260709 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.261561 (+ 852us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:10:24.261708 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0620 20:10:24.261763 (+ 55us) server_negotiation.cc:299] Negotiation successful
0620 20:10:24.261862 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7581,"thread_start_us":91,"threads_started":1}
I20260620 20:10:24.264868 17237 tablet_bootstrap.cc:492] T 947100db3a5c43cebaa04e97c97a503e P 52a87cfecb9e4a569cb3340b709d580c: Bootstrap starting.
I20260620 20:10:24.265010 16878 tablet_service.cc:1467] Processing CreateTablet for tablet 947100db3a5c43cebaa04e97c97a503e (DEFAULT_TABLE table=test-table [id=fac1b29efaf54a7aa3169c18311242ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:10:24.266217 16878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 947100db3a5c43cebaa04e97c97a503e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:10:24.268252 17237 tablet_bootstrap.cc:654] T 947100db3a5c43cebaa04e97c97a503e P 52a87cfecb9e4a569cb3340b709d580c: Neither blocks nor log segments found. Creating new log.
I20260620 20:10:24.273200 17237 log.cc:824] T 947100db3a5c43cebaa04e97c97a503e P 52a87cfecb9e4a569cb3340b709d580c: Log is configured to *not* fsync() on all Append() calls
I20260620 20:10:24.276629 17238 tablet_bootstrap.cc:492] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807: Bootstrap starting.
I20260620 20:10:24.279121 17237 tablet_bootstrap.cc:492] T 947100db3a5c43cebaa04e97c97a503e P 52a87cfecb9e4a569cb3340b709d580c: No bootstrap required, opened a new log
I20260620 20:10:24.279291 17238 tablet_bootstrap.cc:654] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807: Neither blocks nor log segments found. Creating new log.
I20260620 20:10:24.279363 17237 ts_tablet_manager.cc:1397] T 947100db3a5c43cebaa04e97c97a503e P 52a87cfecb9e4a569cb3340b709d580c: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.005s
I20260620 20:10:24.279423 17236 tablet_bootstrap.cc:492] T 947100db3a5c43cebaa04e97c97a503e P 172545da50194bea9c7b4e3103c12138: Bootstrap starting.
I20260620 20:10:24.280179 17238 log.cc:824] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807: Log is configured to *not* fsync() on all Append() calls
I20260620 20:10:24.282827 17238 tablet_bootstrap.cc:492] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807: No bootstrap required, opened a new log
I20260620 20:10:24.283028 17238 ts_tablet_manager.cc:1397] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260620 20:10:24.283417 17236 tablet_bootstrap.cc:654] T 947100db3a5c43cebaa04e97c97a503e P 172545da50194bea9c7b4e3103c12138: Neither blocks nor log segments found. Creating new log.
I20260620 20:10:24.283941 17237 raft_consensus.cc:348] T 947100db3a5c43cebaa04e97c97a503e P 52a87cfecb9e4a569cb3340b709d580c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "172545da50194bea9c7b4e3103c12138" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 35989 } } peers { permanent_uuid: "52a87cfecb9e4a569cb3340b709d580c" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 40601 } } peers { permanent_uuid: "5c410fa48d23428693b41b84b7d70807" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 41631 } }
I20260620 20:10:24.284229 17237 raft_consensus.cc:374] T 947100db3a5c43cebaa04e97c97a503e P 52a87cfecb9e4a569cb3340b709d580c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:10:24.284277 17237 raft_consensus.cc:729] T 947100db3a5c43cebaa04e97c97a503e P 52a87cfecb9e4a569cb3340b709d580c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52a87cfecb9e4a569cb3340b709d580c, State: Initialized, Role: FOLLOWER
I20260620 20:10:24.284744 17236 log.cc:824] T 947100db3a5c43cebaa04e97c97a503e P 172545da50194bea9c7b4e3103c12138: Log is configured to *not* fsync() on all Append() calls
I20260620 20:10:24.285140 17237 consensus_queue.cc:260] T 947100db3a5c43cebaa04e97c97a503e P 52a87cfecb9e4a569cb3340b709d580c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "172545da50194bea9c7b4e3103c12138" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 35989 } } peers { permanent_uuid: "52a87cfecb9e4a569cb3340b709d580c" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 40601 } } peers { permanent_uuid: "5c410fa48d23428693b41b84b7d70807" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 41631 } }
I20260620 20:10:24.286339 17237 ts_tablet_manager.cc:1428] T 947100db3a5c43cebaa04e97c97a503e P 52a87cfecb9e4a569cb3340b709d580c: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260620 20:10:24.286863 17218 heartbeater.cc:503] Master 127.5.91.190:32991 was elected leader, sending a full tablet report...
I20260620 20:10:24.287531 17238 raft_consensus.cc:348] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "172545da50194bea9c7b4e3103c12138" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 35989 } } peers { permanent_uuid: "52a87cfecb9e4a569cb3340b709d580c" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 40601 } } peers { permanent_uuid: "5c410fa48d23428693b41b84b7d70807" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 41631 } }
I20260620 20:10:24.287787 17238 raft_consensus.cc:374] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:10:24.287834 17238 raft_consensus.cc:729] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c410fa48d23428693b41b84b7d70807, State: Initialized, Role: FOLLOWER
I20260620 20:10:24.288318 17238 consensus_queue.cc:260] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "172545da50194bea9c7b4e3103c12138" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 35989 } } peers { permanent_uuid: "52a87cfecb9e4a569cb3340b709d580c" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 40601 } } peers { permanent_uuid: "5c410fa48d23428693b41b84b7d70807" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 41631 } }
I20260620 20:10:24.289616 17238 ts_tablet_manager.cc:1428] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260620 20:10:24.289965 16944 heartbeater.cc:503] Master 127.5.91.190:32991 was elected leader, sending a full tablet report...
I20260620 20:10:24.295578 17236 tablet_bootstrap.cc:492] T 947100db3a5c43cebaa04e97c97a503e P 172545da50194bea9c7b4e3103c12138: No bootstrap required, opened a new log
I20260620 20:10:24.296017 17236 ts_tablet_manager.cc:1397] T 947100db3a5c43cebaa04e97c97a503e P 172545da50194bea9c7b4e3103c12138: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.000s
I20260620 20:10:24.301544 17236 raft_consensus.cc:348] T 947100db3a5c43cebaa04e97c97a503e P 172545da50194bea9c7b4e3103c12138 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "172545da50194bea9c7b4e3103c12138" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 35989 } } peers { permanent_uuid: "52a87cfecb9e4a569cb3340b709d580c" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 40601 } } peers { permanent_uuid: "5c410fa48d23428693b41b84b7d70807" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 41631 } }
I20260620 20:10:24.301996 17236 raft_consensus.cc:374] T 947100db3a5c43cebaa04e97c97a503e P 172545da50194bea9c7b4e3103c12138 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:10:24.302102 17236 raft_consensus.cc:729] T 947100db3a5c43cebaa04e97c97a503e P 172545da50194bea9c7b4e3103c12138 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 172545da50194bea9c7b4e3103c12138, State: Initialized, Role: FOLLOWER
I20260620 20:10:24.302743 17236 consensus_queue.cc:260] T 947100db3a5c43cebaa04e97c97a503e P 172545da50194bea9c7b4e3103c12138 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "172545da50194bea9c7b4e3103c12138" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 35989 } } peers { permanent_uuid: "52a87cfecb9e4a569cb3340b709d580c" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 40601 } } peers { permanent_uuid: "5c410fa48d23428693b41b84b7d70807" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 41631 } }
I20260620 20:10:24.303951 17236 ts_tablet_manager.cc:1428] T 947100db3a5c43cebaa04e97c97a503e P 172545da50194bea9c7b4e3103c12138: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260620 20:10:24.304381 17083 heartbeater.cc:503] Master 127.5.91.190:32991 was elected leader, sending a full tablet report...
Jun 20 20:10:24 dist-test-slave-l8h4 krb5kdc[16252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986223, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for HTTP/127.5.91.148@KRBTEST.COM
I20260620 20:10:24.357678 17243 raft_consensus.cc:482] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:10:24.357883 17243 raft_consensus.cc:504] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "172545da50194bea9c7b4e3103c12138" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 35989 } } peers { permanent_uuid: "52a87cfecb9e4a569cb3340b709d580c" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 40601 } } peers { permanent_uuid: "5c410fa48d23428693b41b84b7d70807" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 41631 } }
I20260620 20:10:24.359326 17243 leader_election.cc:288] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 172545da50194bea9c7b4e3103c12138 (127.5.91.130:35989), 52a87cfecb9e4a569cb3340b709d580c (127.5.91.131:40601)
I20260620 20:10:24.370029 17231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:10:24.360686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:52157 (local address 127.5.91.130:35989)
0620 20:10:24.360801 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0620 20:10:24.360806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:10:24.361442 (+ 636us) server_negotiation.cc:407] Connection header received
0620 20:10:24.361647 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:10:24.361651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:10:24.361698 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:10:24.361742 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:10:24.362455 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.363171 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.367788 (+ 4617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.368314 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.368398 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:10:24.368452 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 20:10:24.369799 (+ 1347us) server_negotiation.cc:299] Negotiation successful
0620 20:10:24.369888 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 20:10:24.370674 17247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:10:24.359681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:35989 (local address 127.5.91.129:52157)
0620 20:10:24.361357 (+ 1676us) negotiation.cc:107] Waiting for socket to connect
0620 20:10:24.361370 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:10:24.361528 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:10:24.361894 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:10:24.361896 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:10:24.361914 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:10:24.362051 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:10:24.362058 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.363318 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.363321 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:10:24.364598 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:10:24.364610 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.368578 (+ 3968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.368582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:10:24.368707 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:10:24.368731 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 20:10:24.370455 (+ 1724us) client_negotiation.cc:240] Negotiation successful
0620 20:10:24.370549 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1593,"thread_start_us":122,"threads_started":1}
I20260620 20:10:24.371469 17233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:10:24.360232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:55451 (local address 127.5.91.131:40601)
0620 20:10:24.360358 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0620 20:10:24.360363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:10:24.362380 (+ 2017us) server_negotiation.cc:407] Connection header received
0620 20:10:24.364866 (+ 2486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:10:24.364870 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:10:24.364925 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:10:24.364971 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:10:24.365354 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.366127 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.368842 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.369443 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.369544 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:10:24.369589 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0620 20:10:24.371269 (+ 1680us) server_negotiation.cc:299] Negotiation successful
0620 20:10:24.371353 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 20:10:24.372416 17036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "947100db3a5c43cebaa04e97c97a503e" candidate_uuid: "5c410fa48d23428693b41b84b7d70807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "172545da50194bea9c7b4e3103c12138" is_pre_election: true
I20260620 20:10:24.372757 17036 raft_consensus.cc:2454] T 947100db3a5c43cebaa04e97c97a503e P 172545da50194bea9c7b4e3103c12138 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c410fa48d23428693b41b84b7d70807 in term 0.
I20260620 20:10:24.373375 16832 leader_election.cc:302] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 172545da50194bea9c7b4e3103c12138, 5c410fa48d23428693b41b84b7d70807; no voters:
I20260620 20:10:24.373685 17243 raft_consensus.cc:2781] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:10:24.373772 17243 raft_consensus.cc:482] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:10:24.373823 17243 raft_consensus.cc:3037] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:10:24.374056 17246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:10:24.360107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:40601 (local address 127.5.91.129:55451)
0620 20:10:24.362301 (+ 2194us) negotiation.cc:107] Waiting for socket to connect
0620 20:10:24.362314 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:10:24.364742 (+ 2428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:10:24.365101 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:10:24.365103 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:10:24.365119 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:10:24.365223 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:10:24.365241 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.366298 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.366301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:10:24.367638 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:10:24.367650 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.370868 (+ 3218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.370876 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:10:24.371122 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:10:24.371147 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 20:10:24.371681 (+ 534us) client_negotiation.cc:240] Negotiation successful
0620 20:10:24.373931 (+ 2250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2128,"thread_start_us":84,"threads_started":1}
I20260620 20:10:24.374872 17169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "947100db3a5c43cebaa04e97c97a503e" candidate_uuid: "5c410fa48d23428693b41b84b7d70807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52a87cfecb9e4a569cb3340b709d580c" is_pre_election: true
I20260620 20:10:24.375130 17243 raft_consensus.cc:504] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "172545da50194bea9c7b4e3103c12138" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 35989 } } peers { permanent_uuid: "52a87cfecb9e4a569cb3340b709d580c" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 40601 } } peers { permanent_uuid: "5c410fa48d23428693b41b84b7d70807" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 41631 } }
I20260620 20:10:24.375206 17169 raft_consensus.cc:2454] T 947100db3a5c43cebaa04e97c97a503e P 52a87cfecb9e4a569cb3340b709d580c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c410fa48d23428693b41b84b7d70807 in term 0.
I20260620 20:10:24.375485 17243 leader_election.cc:288] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807 [CANDIDATE]: Term 1 election: Requested vote from peers 172545da50194bea9c7b4e3103c12138 (127.5.91.130:35989), 52a87cfecb9e4a569cb3340b709d580c (127.5.91.131:40601)
I20260620 20:10:24.376319 17036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "947100db3a5c43cebaa04e97c97a503e" candidate_uuid: "5c410fa48d23428693b41b84b7d70807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "172545da50194bea9c7b4e3103c12138"
I20260620 20:10:24.376435 17036 raft_consensus.cc:3037] T 947100db3a5c43cebaa04e97c97a503e P 172545da50194bea9c7b4e3103c12138 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:10:24.377624 17036 raft_consensus.cc:2454] T 947100db3a5c43cebaa04e97c97a503e P 172545da50194bea9c7b4e3103c12138 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c410fa48d23428693b41b84b7d70807 in term 1.
I20260620 20:10:24.377952 16832 leader_election.cc:302] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 172545da50194bea9c7b4e3103c12138, 5c410fa48d23428693b41b84b7d70807; no voters:
I20260620 20:10:24.377946 17169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "947100db3a5c43cebaa04e97c97a503e" candidate_uuid: "5c410fa48d23428693b41b84b7d70807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52a87cfecb9e4a569cb3340b709d580c"
I20260620 20:10:24.378083 17169 raft_consensus.cc:3037] T 947100db3a5c43cebaa04e97c97a503e P 52a87cfecb9e4a569cb3340b709d580c [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:10:24.378182 17243 raft_consensus.cc:2781] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:10:24.378382 17243 raft_consensus.cc:686] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807 [term 1 LEADER]: Becoming Leader. State: Replica: 5c410fa48d23428693b41b84b7d70807, State: Running, Role: LEADER
I20260620 20:10:24.378731 17243 consensus_queue.cc:237] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "172545da50194bea9c7b4e3103c12138" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 35989 } } peers { permanent_uuid: "52a87cfecb9e4a569cb3340b709d580c" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 40601 } } peers { permanent_uuid: "5c410fa48d23428693b41b84b7d70807" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 41631 } }
I20260620 20:10:24.379349 17169 raft_consensus.cc:2454] T 947100db3a5c43cebaa04e97c97a503e P 52a87cfecb9e4a569cb3340b709d580c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c410fa48d23428693b41b84b7d70807 in term 1.
I20260620 20:10:24.381925 16731 catalog_manager.cc:5495] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c410fa48d23428693b41b84b7d70807 (127.5.91.129). New cstate: current_term: 1 leader_uuid: "5c410fa48d23428693b41b84b7d70807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "172545da50194bea9c7b4e3103c12138" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 35989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52a87cfecb9e4a569cb3340b709d580c" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 40601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c410fa48d23428693b41b84b7d70807" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 41631 } health_report { overall_health: HEALTHY } } }
I20260620 20:10:24.407594 17235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:10:24.403306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39082 (local address 127.5.91.129:41631)
0620 20:10:24.403615 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0620 20:10:24.403619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:10:24.403630 (+ 11us) server_negotiation.cc:407] Connection header received
0620 20:10:24.403750 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:10:24.403752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:10:24.403805 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:10:24.403893 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:10:24.404139 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.404757 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.405774 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:10:24.406085 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:10:24.406203 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:10:24.406522 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:10:24.407012 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:10:24.407114 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 20:10:24.407423 (+ 309us) server_negotiation.cc:299] Negotiation successful
0620 20:10:24.407485 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236}
W20260620 20:10:24.415548 16945 tablet.cc:2369] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:10:24.418398 17169 raft_consensus.cc:1261] T 947100db3a5c43cebaa04e97c97a503e P 52a87cfecb9e4a569cb3340b709d580c [term 1 FOLLOWER]: Refusing update from remote peer 5c410fa48d23428693b41b84b7d70807: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:10:24.418416 17036 raft_consensus.cc:1261] T 947100db3a5c43cebaa04e97c97a503e P 172545da50194bea9c7b4e3103c12138 [term 1 FOLLOWER]: Refusing update from remote peer 5c410fa48d23428693b41b84b7d70807: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260620 20:10:24.418920 17219 tablet.cc:2369] T 947100db3a5c43cebaa04e97c97a503e P 52a87cfecb9e4a569cb3340b709d580c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:10:24.419142 17243 consensus_queue.cc:1035] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "172545da50194bea9c7b4e3103c12138" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 35989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:10:24.424190 17243 consensus_queue.cc:1035] T 947100db3a5c43cebaa04e97c97a503e P 5c410fa48d23428693b41b84b7d70807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52a87cfecb9e4a569cb3340b709d580c" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 40601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:10:24.431200 17252 mvcc.cc:205] Tried to move back new op lower bound from 7299015575209721856 to 7299015575058522112. Current Snapshot: MvccSnapshot[applied={T|T < 7299015575209721856}]
I20260620 20:10:24.443107 16731 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:10:24.443284 16731 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:10:24.445227 17253 mvcc.cc:205] Tried to move back new op lower bound from 7299015575209721856 to 7299015575058522112. Current Snapshot: MvccSnapshot[applied={T|T < 7299015575209721856}]
I20260620 20:10:24.446236 16731 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 353940bd102749c284251bc8044eb29c: Sending DeleteTablet for 3 replicas of tablet 947100db3a5c43cebaa04e97c97a503e
I20260620 20:10:24.447036 17016 tablet_service.cc:1514] Processing DeleteTablet for tablet 947100db3a5c43cebaa04e97c97a503e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:10:24 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:57854
I20260620 20:10:24.448021 17133 tablet_service.cc:1514] Processing DeleteTablet for tablet 947100db3a5c43cebaa04e97c97a503e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:10:24 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:58542
I20260620 20:10:24.448912 16878 tablet_service.cc:1514] Processing DeleteTablet for tablet 947100db3a5c43cebaa04e97c97a503e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:10:24 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:39078
I20260620 20:10:24.450537 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 16812
I20260620 20:10:24.453537 17255 mvcc.cc:205] Tried to move back new op lower bound from 7299015575209721856 to 7299015575058522112. Current Snapshot: MvccSnapshot[applied={T|T < 7299015575209721856}]
I20260620 20:10:24.458807 17261 tablet_replica.cc:330] T 947100db3a5c43cebaa04e97c97a503e P 52a87cfecb9e4a569cb3340b709d580c: stopping tablet replica
I20260620 20:10:24.459139 17261 raft_consensus.cc:2229] T 947100db3a5c43cebaa04e97c97a503e P 52a87cfecb9e4a569cb3340b709d580c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:10:24.459466 17261 raft_consensus.cc:2258] T 947100db3a5c43cebaa04e97c97a503e P 52a87cfecb9e4a569cb3340b709d580c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:10:24.459808 17260 tablet_replica.cc:330] T 947100db3a5c43cebaa04e97c97a503e P 172545da50194bea9c7b4e3103c12138: stopping tablet replica
I20260620 20:10:24.460150 17260 raft_consensus.cc:2229] T 947100db3a5c43cebaa04e97c97a503e P 172545da50194bea9c7b4e3103c12138 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:10:24.460484 17260 raft_consensus.cc:2258] T 947100db3a5c43cebaa04e97c97a503e P 172545da50194bea9c7b4e3103c12138 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:10:24.462028 17261 ts_tablet_manager.cc:1905] T 947100db3a5c43cebaa04e97c97a503e P 52a87cfecb9e4a569cb3340b709d580c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:10:24.464825 17261 ts_tablet_manager.cc:1918] T 947100db3a5c43cebaa04e97c97a503e P 52a87cfecb9e4a569cb3340b709d580c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:10:24.464960 17261 log.cc:1196] T 947100db3a5c43cebaa04e97c97a503e P 52a87cfecb9e4a569cb3340b709d580c: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/947100db3a5c43cebaa04e97c97a503e
W20260620 20:10:24.464993 17107 connection.cc:535] server connection from 127.5.91.129:55451 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 20:10:24.465618 17261 ts_tablet_manager.cc:1939] T 947100db3a5c43cebaa04e97c97a503e P 52a87cfecb9e4a569cb3340b709d580c: Deleting consensus metadata
I20260620 20:10:24.467016 16717 catalog_manager.cc:4841] TS 52a87cfecb9e4a569cb3340b709d580c (127.5.91.131:40601): tablet 947100db3a5c43cebaa04e97c97a503e (table test-table [id=fac1b29efaf54a7aa3169c18311242ba]) successfully deleted
W20260620 20:10:24.467227 16717 connection.cc:535] server connection from 127.5.91.129:46487 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 20:10:24.467396 16717 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.91.129:41631 (error 108)
I20260620 20:10:24.467409 17260 ts_tablet_manager.cc:1905] T 947100db3a5c43cebaa04e97c97a503e P 172545da50194bea9c7b4e3103c12138: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:10:24.470021 17260 ts_tablet_manager.cc:1918] T 947100db3a5c43cebaa04e97c97a503e P 172545da50194bea9c7b4e3103c12138: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:10:24.470122 17260 log.cc:1196] T 947100db3a5c43cebaa04e97c97a503e P 172545da50194bea9c7b4e3103c12138: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781986111135528-5486-0/minicluster-data/ts-1/wal/wals/947100db3a5c43cebaa04e97c97a503e
I20260620 20:10:24.470362 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 16952
I20260620 20:10:24.472625 17232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:10:24.472316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:41631 (local address 127.0.0.1:39090)
0620 20:10:24.472460 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 20:10:24.472537 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.129:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260620 20:10:24.472805 16717 catalog_manager.cc:4584] TS 5c410fa48d23428693b41b84b7d70807 (127.5.91.129:41631): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 947100db3a5c43cebaa04e97c97a503e: Network error: Client connection negotiation failed: client connection to 127.5.91.129:41631: connect: Connection refused (error 111)
I20260620 20:10:24.476598 17232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:10:24.476278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:35989 (local address 127.0.0.1:57868)
0620 20:10:24.476445 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 20:10:24.476497 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.130:35989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260620 20:10:24.478348 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 17088
I20260620 20:10:24.489190 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 16699
I20260620 20:10:24.496260 5486 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260620 20:10:24.832134 5486 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260620 20:10:24.832259 5486 mini_ranger.cc:66] Stopping Ranger...
I20260620 20:10:25.167335 5486 mini_ranger.cc:68] Stopped Ranger
2026-06-20 20:10:25.167 UTC [16282] LOG: received smart shutdown request
2026-06-20 20:10:25.171 UTC [16282] LOG: background worker "logical replication launcher" (PID 16291) exited with exit code 1
2026-06-20 20:10:25.171 UTC [16286] LOG: shutting down
2026-06-20 20:10:25.379 UTC [16282] LOG: database system is shut down
2026-06-20T20:10:25Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50612 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:10:25 dist-test-slave-l8h4 krb5kdc[17273](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:10:25 dist-test-slave-l8h4 krb5kdc[17273](info): set up 2 sockets
Jun 20 20:10:25 dist-test-slave-l8h4 krb5kdc[17273](info): commencing operation
krb5kdc: starting...
W20260620 20:10:26.464042 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.971s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:10:26 dist-test-slave-l8h4 krb5kdc[17273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:10:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:10:26Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.91.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.91.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.91.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/rangeradmin_127.5.91.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.91.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/rangeradmin_127.5.91.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.91.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.91.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.91.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/rangerlookup_127.5.91.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.91.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/rangerlookup_127.5.91.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.91.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.91.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.91.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/HTTP_127.5.91.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.91.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/HTTP_127.5.91.148@KRBTEST.COM.keytab.
I20260620 20:10:26.649395 5486 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskSTSRqO/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/postgres -l logfile start
2026-06-20 20:10:29.566 UTC [17303] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-20 20:10:29.566 UTC [17303] LOG: listening on IPv4 address "127.5.91.148", port 52215
2026-06-20 20:10:29.571 UTC [17303] LOG: listening on Unix socket "/tmp/.s.PGSQL.52215"
2026-06-20 20:10:29.592 UTC [17306] LOG: database system was shut down at 2026-06-20 20:10:27 UTC
2026-06-20 20:10:29.596 UTC [17303] LOG: database system is ready to accept connections
I20260620 20:10:30.494796 5486 mini_postgres.cc:96] Postgres bound to 52215
2026-06-20 20:10:30.501 UTC [17314] FATAL: database "slave" does not exist
127.5.91.148:52215 - accepting connections
I20260620 20:10:30.502305 5486 mini_ranger.cc:161] Starting Ranger...
I20260620 20:10:30.517103 5486 mini_ranger.cc:84] Created miniranger Postgres user
I20260620 20:10:31.145679 5486 mini_ranger.cc:87] Created ranger Postgres database
I20260620 20:10:31.145771 5486 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ranger-admin
2026-06-20 20:10:31,457 [I] DB FLAVOR :POSTGRES
2026-06-20 20:10:31,457 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 20:10:31,457 [I] Checking connection..
2026-06-20 20:10:31,457 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:52215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-20 20:10:31,722 [I] Checking connection passed.
2026-06-20 20:10:31,723 [I] --------- Verifying version history table ---------
2026-06-20 20:10:31,723 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:52215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 20:10:32,001 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 20:10:32,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:52215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 20:10:32,290 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 20:10:32,290 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-20 20:10:32,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:52215/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-20 20:10:32.558 UTC [17379] WARNING: there is no transaction in progress
2026-06-20 20:10:32,565 [I] create_dbversion_catalog.sql file imported successfully
2026-06-20 20:10:32,565 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:52215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 20:10:32,834 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-20 20:10:32,834 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 20:10:32,834 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:52215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-20 20:10:33,084 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:52215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-20 20:10:33,369 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:52215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-l8h4.c.gcp-upstream.internal','N') ;"
2026-06-20 20:10:33,645 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:52215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 20:10:33,945 [I] Table x_portal_user does not exist in database ranger
2026-06-20 20:10:33,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:52215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 20:10:34,217 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-20 20:10:34,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:52215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 20:10:34,480 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:52215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 20:10:34,729 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-20 20:10:34,729 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:52215/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-20 20:10:35.849 UTC [17487] WARNING: there is no transaction in progress
2026-06-20 20:10:35.871 UTC [17487] WARNING: there is no transaction in progress
2026-06-20 20:10:35.891 UTC [17487] WARNING: there is no transaction in progress
2026-06-20 20:10:35.910 UTC [17487] WARNING: there is no transaction in progress
2026-06-20 20:10:35.935 UTC [17487] WARNING: there is no transaction in progress
2026-06-20 20:10:35.958 UTC [17487] WARNING: there is no transaction in progress
2026-06-20 20:10:36.026 UTC [17487] WARNING: there is no transaction in progress
2026-06-20 20:10:36.034 UTC [17487] WARNING: there is no transaction in progress
2026-06-20 20:10:36.053 UTC [17487] WARNING: there is no transaction in progress
2026-06-20 20:10:36.065 UTC [17487] WARNING: there is no transaction in progress
2026-06-20 20:10:36.078 UTC [17487] WARNING: there is no transaction in progress
2026-06-20 20:10:36.096 UTC [17487] WARNING: there is no transaction in progress
2026-06-20 20:10:36.107 UTC [17487] WARNING: there is no transaction in progress
2026-06-20 20:10:36.114 UTC [17487] WARNING: there is no transaction in progress
2026-06-20 20:10:36.123 UTC [17487] WARNING: there is no transaction in progress
2026-06-20 20:10:36.130 UTC [17487] WARNING: there is no transaction in progress
2026-06-20 20:10:37,020 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-20 20:10:37,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:52215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-20 20:10:37,279 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-20 20:10:37,279 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:52215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 20:10:37,553 [I] Table x_portal_user already exists in database 'ranger'
2026-06-20 20:10:37,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:52215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 20:10:37,808 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-20 20:10:37,809 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:52215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 20:10:38,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:52215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 20:10:38,324 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:52215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-l8h4.c.gcp-upstream.internal';"
2026-06-20 20:10:38,556 [I] CORE_DB_SCHEMA import status has been updated
2026-06-20 20:10:38,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:52215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-20 20:10:38,793 [I] DB_PATCHES have already been applied
I20260620 20:10:38.798584 5486 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ranger-admin:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskSTSRqO/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260620 20:10:38.798658 5486 mini_ranger.cc:193] Using host: 127.5.91.148
I20260620 20:10:38.801952 5486 mini_ranger.cc:227] Ranger admin URL: http://127.5.91.148:48593
Jun 20, 2026 8:10:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 8:10:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 8:10:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ranger-admin/ews/webapp
Jun 20, 2026 8:10:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 20, 2026 8:10:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 20, 2026 8:10:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ranger-admin/ews/webapp].
Jun 20, 2026 8:10:39 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260620 20:10:39.792096 5486 mini_ranger.cc:160] Time spent starting Ranger: real 9.290s user 0.002s sys 0.008s
Jun 20, 2026 8:11:01 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 20, 2026 8:11:01 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Jun 20, 2026 8:11:01 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 20, 2026 8:11:02 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 20, 2026 8:11:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 20, 2026 8:11:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 20, 2026 8:11:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 20, 2026 8:11:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 20, 2026 8:11:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 20, 2026 8:11:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 20, 2026 8:11:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 20, 2026 8:11:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 20, 2026 8:11:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 20, 2026 8:11:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 20, 2026 8:11:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 20, 2026 8:11:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 20, 2026 8:11:02 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 20, 2026 8:11:02 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260620 20:11:03.557915 5486 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.91.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.91.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.91.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/rangerkms_127.5.91.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.91.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/rangerkms_127.5.91.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.91.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.91.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.91.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/HTTP_127.5.91.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.91.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/HTTP_127.5.91.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 20 20:11:03 dist-test-slave-l8h4 krb5kdc[17273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986263, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260620 20:11:03.642666 5486 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260620 20:11:03.651859 5486 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260620 20:11:04.649421 5486 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260620 20:11:04.649492 5486 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ranger-kms
I20260620 20:11:04.649503 5486 mini_ranger_kms.cc:227] Using postgres at 127.5.91.148:52215
2026-06-20 20:11:04,803 [I] DB FLAVOR :POSTGRES
2026-06-20 20:11:04,803 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 20:11:04,803 [I] Checking connection
2026-06-20 20:11:04,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:52215/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-20 20:11:05,052 [I] connection success
2026-06-20 20:11:05,052 [I] --------- Verifying Ranger DB tables ---------
2026-06-20 20:11:05,052 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-20 20:11:05,052 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:52215/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-20 20:11:05,300 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-20 20:11:05,300 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 20:11:05,300 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-20 20:11:05,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.91.148:52215/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-20 20:11:05,599 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260620 20:11:06.131035 5486 mini_ranger_kms.cc:310] Created kms service
I20260620 20:11:06.362022 5486 mini_ranger_kms.cc:326] Created kudu user
I20260620 20:11:06.403074 5486 mini_ranger_kms.cc:343] Created rangerkms user
Jun 20 20:11:06 dist-test-slave-l8h4 krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986263, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.91.148@KRBTEST.COM
I20260620 20:11:06.833549 5486 mini_ranger_kms.cc:384] Added ranger policy
I20260620 20:11:06.833694 5486 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ranger-kms:/tmp/dist-test-taskSTSRqO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskSTSRqO/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260620 20:11:06.833715 5486 mini_ranger_kms.cc:240] Using host: 127.5.91.148
I20260620 20:11:06.838012 5486 mini_ranger_kms.cc:276] Ranger KMS PID: 17669
I20260620 20:11:06.838099 5486 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.91.148:37877
Jun 20, 2026 8:11:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 8:11:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 8:11:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 20, 2026 8:11:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 20, 2026 8:11:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ranger-kms/ews/webapp].
Jun 20, 2026 8:11:08 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260620 20:11:08.910663 5486 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.268s user 0.013s sys 0.125s
I20260620 20:11:08.910794 5486 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260620 20:11:08.910864 5486 mini_ranger_kms.cc:401] 127.5.91.148:37877/kms/v1/keys
Jun 20 20:11:08 dist-test-slave-l8h4 krb5kdc[17273](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781986268, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.91.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 20 20:11:09 dist-test-slave-l8h4 krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986268, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.91.148@KRBTEST.COM for HTTP/127.5.91.148@KRBTEST.COM
Jun 20 20:11:09 dist-test-slave-l8h4 krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986268, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.91.148@KRBTEST.COM for HTTP/127.5.91.148@KRBTEST.COM
Jun 20 20:11:12 dist-test-slave-l8h4 krb5kdc[17273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:11:12.259179 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:40079
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:42053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:40079
--ranger_config_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.91.148:37877/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:11:12.376163 17719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:12.376436 17719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:12.376477 17719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:12.380652 17719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:11:12.380716 17719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:12.380734 17719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:12.380750 17719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:11:12.380764 17719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:11:12.386590 17719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:42053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.91.148:37877/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:40079
--ranger_config_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:40079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:12.387625 17719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:12.388620 17719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:12.397802 17724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:12.397802 17725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:12.397855 17727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:12.397776 17719 server_base.cc:1034] running on GCE node
I20260620 20:11:12.398600 17719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:12.399652 17719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:12.400830 17719 hybrid_clock.cc:648] HybridClock initialized: now 1781986272400814 us; error 33 us; skew 500 ppm
Jun 20 20:11:12 dist-test-slave-l8h4 krb5kdc[17273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986272, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:12.403874 17719 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:11:12.404968 17719 webserver.cc:460] Webserver started at http://127.5.91.190:37497/ using document root <none> and password file <none>
I20260620 20:11:12.405494 17719 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:12.405550 17719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:12.405732 17719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 20:11:12 dist-test-slave-l8h4 krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986272, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for HTTP/127.5.91.148@KRBTEST.COM
I20260620 20:11:12.472590 17719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "5795af9ec2544ce988385c4671f4cd8e"
format_stamp: "Formatted at 2026-06-20 20:11:12 on dist-test-slave-l8h4"
server_key: "efea7ce3421e27fc1b25c31b378e2548"
server_key_iv: "a36a7098944db97545eafa0f82116e61"
server_key_version: "kuduclusterkey@0"
I20260620 20:11:12.473239 17719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "5795af9ec2544ce988385c4671f4cd8e"
format_stamp: "Formatted at 2026-06-20 20:11:12 on dist-test-slave-l8h4"
server_key: "efea7ce3421e27fc1b25c31b378e2548"
server_key_iv: "a36a7098944db97545eafa0f82116e61"
server_key_version: "kuduclusterkey@0"
I20260620 20:11:12.476898 17719 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260620 20:11:12.495167 17735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:12.496850 17719 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 20:11:12.496996 17719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "5795af9ec2544ce988385c4671f4cd8e"
format_stamp: "Formatted at 2026-06-20 20:11:12 on dist-test-slave-l8h4"
server_key: "efea7ce3421e27fc1b25c31b378e2548"
server_key_iv: "a36a7098944db97545eafa0f82116e61"
server_key_version: "kuduclusterkey@0"
I20260620 20:11:12.497149 17719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:12.513437 17719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:12.517554 17719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:12.517757 17719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:12.526535 17787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:40079 every 8 connection(s)
I20260620 20:11:12.526541 17719 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:40079
I20260620 20:11:12.527772 17719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:11:12.531160 17788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:11:12.536911 17788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e: Bootstrap starting.
I20260620 20:11:12.537182 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 17719
I20260620 20:11:12.537297 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:11:12.537590 5486 external_mini_cluster.cc:1428] Setting key c5c056c968340dd6310fe9311da40f62
I20260620 20:11:12.539666 17788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e: Neither blocks nor log segments found. Creating new log.
I20260620 20:11:12.540381 17788 log.cc:824] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e: Log is configured to *not* fsync() on all Append() calls
I20260620 20:11:12.542467 17788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e: No bootstrap required, opened a new log
I20260620 20:11:12.546319 17788 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5795af9ec2544ce988385c4671f4cd8e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40079 } }
I20260620 20:11:12.546573 17788 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:11:12.546645 17788 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5795af9ec2544ce988385c4671f4cd8e, State: Initialized, Role: FOLLOWER
I20260620 20:11:12.547089 17788 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5795af9ec2544ce988385c4671f4cd8e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40079 } }
I20260620 20:11:12.547219 17788 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:11:12.547297 17788 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:11:12.547406 17788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [term 0 FOLLOWER]: Advancing to term 1
Jun 20 20:11:12 dist-test-slave-l8h4 krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:11:12.548511 17788 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5795af9ec2544ce988385c4671f4cd8e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40079 } }
I20260620 20:11:12.548908 17788 leader_election.cc:302] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5795af9ec2544ce988385c4671f4cd8e; no voters:
I20260620 20:11:12.549276 17788 leader_election.cc:288] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:11:12.549358 17793 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:11:12.549589 17793 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [term 1 LEADER]: Becoming Leader. State: Replica: 5795af9ec2544ce988385c4671f4cd8e, State: Running, Role: LEADER
I20260620 20:11:12.549935 17793 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5795af9ec2544ce988385c4671f4cd8e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40079 } }
I20260620 20:11:12.550730 17788 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:11:12.551195 17793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5795af9ec2544ce988385c4671f4cd8e. Latest consensus state: current_term: 1 leader_uuid: "5795af9ec2544ce988385c4671f4cd8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5795af9ec2544ce988385c4671f4cd8e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40079 } } }
I20260620 20:11:12.551393 17793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [sys.catalog]: This master's current role is: LEADER
I20260620 20:11:12.551739 17793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5795af9ec2544ce988385c4671f4cd8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5795af9ec2544ce988385c4671f4cd8e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40079 } } }
I20260620 20:11:12.551900 17793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [sys.catalog]: This master's current role is: LEADER
I20260620 20:11:12.552223 17798 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:11:12.555111 17791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:12.538983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38964 (local address 127.5.91.190:40079)
0620 20:11:12.539491 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:12.539510 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:12.539553 (+ 43us) server_negotiation.cc:407] Connection header received
0620 20:11:12.540409 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:12.540433 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:12.540874 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:12.541228 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:12.542023 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:12.543617 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:12.544936 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:12.545907 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:12.546077 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:12.549025 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:12.549032 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:12.549050 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:12.549084 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:12.551293 (+ 2209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:12.551834 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:12.551840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:12.551846 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:12.551973 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:12.552265 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:12.552268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:12.552270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:12.552825 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:12.553704 (+ 879us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:12.553851 (+ 147us) server_negotiation.cc:299] Negotiation successful
0620 20:11:12.554131 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":149,"threads_started":1}
I20260620 20:11:12.555666 17798 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:11:12.556084 17788 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 20:11:12.573004 17798 catalog_manager.cc:1347] Generated new cluster ID: 8921691ee4774ecbbefea84979e2d8ce
I20260620 20:11:12.573088 17798 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:11:12.616533 17798 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:11:12.618029 17798 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:11:12.626339 17798 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e: Generated new TSK 0
I20260620 20:11:12.627321 17798 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.91.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
I20260620 20:11:13.343183 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.91.148:37877/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:40079
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:42053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:11:13.489760 17833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:13.490065 17833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:13.490129 17833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:13.495750 17833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:13.495814 17833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:13.495898 17833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:11:13.502758 17833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:42053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.91.148:37877/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:40079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:13.504192 17833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:13.505505 17833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:13.516556 17841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:13.516727 17833 server_base.cc:1034] running on GCE node
W20260620 20:11:13.516861 17840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:13.516865 17843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:13.517400 17833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:13.518088 17833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:13.519310 17833 hybrid_clock.cc:648] HybridClock initialized: now 1781986273519289 us; error 36 us; skew 500 ppm
Jun 20 20:11:13 dist-test-slave-l8h4 krb5kdc[17273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986273, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:13.522082 17833 init.cc:377] Logged in from keytab as kudu/127.5.91.129@KRBTEST.COM (short username kudu)
I20260620 20:11:13.523036 17833 webserver.cc:460] Webserver started at http://127.5.91.129:44407/ using document root <none> and password file <none>
I20260620 20:11:13.523573 17833 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:13.523638 17833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:13.523881 17833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 20:11:13 dist-test-slave-l8h4 krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986273, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for HTTP/127.5.91.148@KRBTEST.COM
I20260620 20:11:13.539269 17833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "8a3519cbf7934273b81427b3d2cd57c5"
format_stamp: "Formatted at 2026-06-20 20:11:13 on dist-test-slave-l8h4"
server_key: "1aeed0c44707342d7b8b27ecaed833f2"
server_key_iv: "d202ffbfd6509a2a170ecae5b88aaeef"
server_key_version: "kuduclusterkey@0"
I20260620 20:11:13.539958 17833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "8a3519cbf7934273b81427b3d2cd57c5"
format_stamp: "Formatted at 2026-06-20 20:11:13 on dist-test-slave-l8h4"
server_key: "1aeed0c44707342d7b8b27ecaed833f2"
server_key_iv: "d202ffbfd6509a2a170ecae5b88aaeef"
server_key_version: "kuduclusterkey@0"
I20260620 20:11:13.547555 17833 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260620 20:11:13.570209 17850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:13.584421 17833 fs_manager.cc:725] Time spent opening block manager: real 0.019s user 0.002s sys 0.000s
I20260620 20:11:13.584590 17833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "8a3519cbf7934273b81427b3d2cd57c5"
format_stamp: "Formatted at 2026-06-20 20:11:13 on dist-test-slave-l8h4"
server_key: "1aeed0c44707342d7b8b27ecaed833f2"
server_key_iv: "d202ffbfd6509a2a170ecae5b88aaeef"
server_key_version: "kuduclusterkey@0"
I20260620 20:11:13.584736 17833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:13.597956 17833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:13.602684 17833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:13.603019 17833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:13.603727 17833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:11:13.604635 17833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:11:13.604686 17833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:13.604738 17833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:11:13.604771 17833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:13.615562 17833 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:44283
I20260620 20:11:13.615597 17966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:44283 every 8 connection(s)
I20260620 20:11:13.616644 17833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:11:13.625604 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 17833
I20260620 20:11:13.625725 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:11:13.626006 5486 external_mini_cluster.cc:1428] Setting key 30c4faee6d2d1e0751a10dc684f219d8
Jun 20 20:11:13 dist-test-slave-l8h4 krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986273, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:11:13.637091 17969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:13.618569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:59057 (local address 127.5.91.190:40079)
0620 20:11:13.618818 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:13.618825 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:13.619646 (+ 821us) server_negotiation.cc:407] Connection header received
0620 20:11:13.620603 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:13.620606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:13.620664 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:13.620746 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:13.622187 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:13.623051 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:13.624761 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:13.625139 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:13.625222 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:13.632787 (+ 7565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:13.632791 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:13.632794 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:13.632824 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:13.634606 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:13.635166 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:13.635169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:13.635171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:13.635260 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:13.635662 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:13.635666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:13.635667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:13.635894 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:13.635981 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:13.636769 (+ 788us) server_negotiation.cc:299] Negotiation successful
0620 20:11:13.636906 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":75,"threads_started":1}
I20260620 20:11:13.638717 17970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:13.619009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:40079 (local address 127.5.91.129:59057)
0620 20:11:13.619502 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:13.619551 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:13.620386 (+ 835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:13.620932 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:13.620941 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:13.621291 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:13.621976 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:13.621999 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:13.623211 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:13.623214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:13.624622 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:13.624633 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:13.629830 (+ 5197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:13.629834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:13.630070 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:13.630819 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:13.630840 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:13.632577 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:13.634726 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:13.634732 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:13.634749 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:13.635069 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:13.635368 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:13.635372 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:13.635374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:13.635562 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:13.636046 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:13.636053 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:13.636413 (+ 360us) client_negotiation.cc:769] Sending connection context
0620 20:11:13.637274 (+ 861us) client_negotiation.cc:240] Negotiation successful
0620 20:11:13.637641 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":120,"threads_started":1}
I20260620 20:11:13.640187 17967 heartbeater.cc:347] Connected to a master server at 127.5.91.190:40079
I20260620 20:11:13.640439 17967 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:13.641031 17967 heartbeater.cc:511] Master 127.5.91.190:40079 requested a full tablet report, sending...
I20260620 20:11:13.642832 17752 ts_manager.cc:194] Registered new tserver with Master: 8a3519cbf7934273b81427b3d2cd57c5 (127.5.91.129:44283)
I20260620 20:11:13.644291 17752 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:59057
WARNING: no policy specified for kudu/127.5.91.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
I20260620 20:11:13.670413 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.91.148:37877/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:40079
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:42053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 20:11:13 dist-test-slave-l8h4 krb5kdc[17273](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781986273, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260620 20:11:13.830821 17973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:13.831254 17973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:13.831341 17973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:13.837651 17973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:13.837790 17973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:13.837908 17973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:11:13.846520 17973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:42053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.91.148:37877/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:40079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:13.847865 17973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:13.849205 17973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:13.861294 17982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:13.861503 17980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:13.861449 17979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:13.861893 17973 server_base.cc:1034] running on GCE node
I20260620 20:11:13.862396 17973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:13.863214 17973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:13.864414 17973 hybrid_clock.cc:648] HybridClock initialized: now 1781986273864390 us; error 40 us; skew 500 ppm
Jun 20 20:11:13 dist-test-slave-l8h4 krb5kdc[17273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986273, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:13.868060 17973 init.cc:377] Logged in from keytab as kudu/127.5.91.130@KRBTEST.COM (short username kudu)
I20260620 20:11:13.869277 17973 webserver.cc:460] Webserver started at http://127.5.91.130:33453/ using document root <none> and password file <none>
I20260620 20:11:13.870002 17973 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:13.870086 17973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:13.870401 17973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 20:11:13 dist-test-slave-l8h4 krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986273, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for HTTP/127.5.91.148@KRBTEST.COM
I20260620 20:11:13.902040 17973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "2bd4539212d544dfa0f5ae9b90b4b8b7"
format_stamp: "Formatted at 2026-06-20 20:11:13 on dist-test-slave-l8h4"
server_key: "34c03609716b7b890654e450ba308001"
server_key_iv: "e178949b97ba4675b674dd73c231701b"
server_key_version: "kuduclusterkey@0"
I20260620 20:11:13.902854 17973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "2bd4539212d544dfa0f5ae9b90b4b8b7"
format_stamp: "Formatted at 2026-06-20 20:11:13 on dist-test-slave-l8h4"
server_key: "34c03609716b7b890654e450ba308001"
server_key_iv: "e178949b97ba4675b674dd73c231701b"
server_key_version: "kuduclusterkey@0"
I20260620 20:11:13.907948 17973 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260620 20:11:13.926952 17989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:13.928833 17973 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260620 20:11:13.929085 17973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "2bd4539212d544dfa0f5ae9b90b4b8b7"
format_stamp: "Formatted at 2026-06-20 20:11:13 on dist-test-slave-l8h4"
server_key: "34c03609716b7b890654e450ba308001"
server_key_iv: "e178949b97ba4675b674dd73c231701b"
server_key_version: "kuduclusterkey@0"
I20260620 20:11:13.929317 17973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:13.970212 17973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:13.973170 17973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:13.973424 17973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:13.974229 17973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:11:13.975271 17973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:11:13.975334 17973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:13.975432 17973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:11:13.975461 17973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:13.987247 17973 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:41615
I20260620 20:11:13.988204 17973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:11:13.988950 18103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:41615 every 8 connection(s)
I20260620 20:11:13.992306 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 17973
I20260620 20:11:13.992460 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:11:13.992875 5486 external_mini_cluster.cc:1428] Setting key 1eea1c235b4151a32c7ece7a901aaa2b
Jun 20 20:11:13 dist-test-slave-l8h4 krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986273, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:11:14.005924 18106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:13.990517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:40079 (local address 127.5.91.130:51443)
0620 20:11:13.991005 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:13.991054 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:13.992013 (+ 959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:13.992908 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:13.992918 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:13.993274 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:13.993881 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:13.993895 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:13.994960 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:13.994963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:13.996709 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:13.996723 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:13.997379 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:13.997382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:13.997530 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:13.998125 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:13.998180 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:13.999776 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:14.001783 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:14.001790 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:14.001810 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:14.002423 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:14.002848 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:14.002851 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:14.002853 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:14.003062 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:14.003751 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:14.003760 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:14.004185 (+ 425us) client_negotiation.cc:769] Sending connection context
0620 20:11:14.004409 (+ 224us) client_negotiation.cc:240] Negotiation successful
0620 20:11:14.004798 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":119,"threads_started":1}
I20260620 20:11:14.006738 17969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:13.990080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:51443 (local address 127.5.91.190:40079)
0620 20:11:13.990238 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:13.990242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:13.991146 (+ 904us) server_negotiation.cc:407] Connection header received
0620 20:11:13.992255 (+ 1109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:13.992259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:13.992319 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:13.992371 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:13.994092 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:13.994793 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:13.996864 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:13.997128 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:13.997188 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:13.999975 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:13.999979 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:13.999982 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:14.000012 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:14.001649 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:14.002551 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:14.002555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:14.002557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:14.002686 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:14.003294 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:14.003298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:14.003300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:14.003571 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:14.003691 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:14.006432 (+ 2741us) server_negotiation.cc:299] Negotiation successful
0620 20:11:14.006579 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
WARNING: no policy specified for kudu/127.5.91.131@KRBTEST.COM; defaulting to no policy
I20260620 20:11:14.008406 18104 heartbeater.cc:347] Connected to a master server at 127.5.91.190:40079
I20260620 20:11:14.008658 18104 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:14.009232 18104 heartbeater.cc:511] Master 127.5.91.190:40079 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.131@KRBTEST.COM" created.
I20260620 20:11:14.011970 17752 ts_manager.cc:194] Registered new tserver with Master: 2bd4539212d544dfa0f5ae9b90b4b8b7 (127.5.91.130:41615)
I20260620 20:11:14.012941 17752 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:51443
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
I20260620 20:11:14.038365 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.91.148:37877/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:40079
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:42053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 20:11:14 dist-test-slave-l8h4 krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986273, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for HTTP/127.5.91.148@KRBTEST.COM
W20260620 20:11:14.200860 18109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:14.201249 18109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:14.201376 18109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:14.207588 18109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:14.207667 18109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:14.207754 18109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:11:14.215672 18109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:42053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.91.148:37877/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:40079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:14.217096 18109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:14.218750 18109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:14.232298 18117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:14.232564 18114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:14.233206 18115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:14.233513 18109 server_base.cc:1034] running on GCE node
I20260620 20:11:14.234093 18109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:14.234902 18109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:14.242295 18109 hybrid_clock.cc:648] HybridClock initialized: now 1781986274242246 us; error 54 us; skew 500 ppm
Jun 20 20:11:14 dist-test-slave-l8h4 krb5kdc[17273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986274, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:14.246214 18109 init.cc:377] Logged in from keytab as kudu/127.5.91.131@KRBTEST.COM (short username kudu)
I20260620 20:11:14.247551 18109 webserver.cc:460] Webserver started at http://127.5.91.131:45811/ using document root <none> and password file <none>
I20260620 20:11:14.248304 18109 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:14.248399 18109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:14.248629 18109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 20:11:14 dist-test-slave-l8h4 krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986274, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for HTTP/127.5.91.148@KRBTEST.COM
I20260620 20:11:14.263760 18109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "3b5dacd48f744f4989426a1c1129f244"
format_stamp: "Formatted at 2026-06-20 20:11:14 on dist-test-slave-l8h4"
server_key: "fb171cc3b9aeb7bb54a4df709197f7ea"
server_key_iv: "8019ef22a644c14025bb7287df06f4b0"
server_key_version: "kuduclusterkey@0"
I20260620 20:11:14.264359 18109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "3b5dacd48f744f4989426a1c1129f244"
format_stamp: "Formatted at 2026-06-20 20:11:14 on dist-test-slave-l8h4"
server_key: "fb171cc3b9aeb7bb54a4df709197f7ea"
server_key_iv: "8019ef22a644c14025bb7287df06f4b0"
server_key_version: "kuduclusterkey@0"
I20260620 20:11:14.271803 18109 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260620 20:11:14.294876 18124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:14.296631 18109 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.002s sys 0.000s
I20260620 20:11:14.296779 18109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "3b5dacd48f744f4989426a1c1129f244"
format_stamp: "Formatted at 2026-06-20 20:11:14 on dist-test-slave-l8h4"
server_key: "fb171cc3b9aeb7bb54a4df709197f7ea"
server_key_iv: "8019ef22a644c14025bb7287df06f4b0"
server_key_version: "kuduclusterkey@0"
I20260620 20:11:14.296926 18109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:14.316351 18109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:14.319931 18109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:14.320109 18109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:14.320773 18109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:11:14.321714 18109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:11:14.321759 18109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:14.321808 18109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:11:14.321827 18109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:14.332216 18109 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:46361
I20260620 20:11:14.332598 18238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:46361 every 8 connection(s)
I20260620 20:11:14.333307 18109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:11:14.343531 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 18109
I20260620 20:11:14.343657 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:11:14.343920 5486 external_mini_cluster.cc:1428] Setting key d13d36e993849d917e8ef55abbbdddc0
Jun 20 20:11:14 dist-test-slave-l8h4 krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986274, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:11:14.349974 17969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:14.337104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:44723 (local address 127.5.91.190:40079)
0620 20:11:14.337220 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:14.337225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:14.337242 (+ 17us) server_negotiation.cc:407] Connection header received
0620 20:11:14.338358 (+ 1116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:14.338361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:14.338418 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:14.338514 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:14.339977 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:14.340798 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:14.342528 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:14.342857 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:14.342964 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:14.345805 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:14.345810 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:14.345812 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:14.345841 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:14.347621 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:14.348207 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:14.348211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:14.348212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:14.348297 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:14.348665 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:14.348668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:14.348670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:14.348898 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:14.349000 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:14.349652 (+ 652us) server_negotiation.cc:299] Negotiation successful
0620 20:11:14.349794 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 20:11:14.351200 18241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:14.335977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:40079 (local address 127.5.91.131:44723)
0620 20:11:14.336551 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:14.336610 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:14.338088 (+ 1478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:14.338715 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:14.338725 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:14.339103 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:14.339803 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:14.339825 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:14.340973 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:14.340977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:14.342396 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:14.342408 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:14.343027 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:14.343031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:14.343257 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:14.343949 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:14.343971 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:14.345603 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:14.347744 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:14.347751 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:14.347768 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:14.348110 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:14.348381 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:14.348384 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:14.348386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:14.348576 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:14.348999 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:14.349006 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:14.349333 (+ 327us) client_negotiation.cc:769] Sending connection context
0620 20:11:14.349587 (+ 254us) client_negotiation.cc:240] Negotiation successful
0620 20:11:14.349940 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":147,"threads_started":1}
I20260620 20:11:14.352339 18239 heartbeater.cc:347] Connected to a master server at 127.5.91.190:40079
I20260620 20:11:14.352597 18239 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:14.353181 18239 heartbeater.cc:511] Master 127.5.91.190:40079 requested a full tablet report, sending...
I20260620 20:11:14.354358 17752 ts_manager.cc:194] Registered new tserver with Master: 3b5dacd48f744f4989426a1c1129f244 (127.5.91.131:46361)
I20260620 20:11:14.355152 17752 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:44723
I20260620 20:11:14.358963 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 20:11:14.359772 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 17833
I20260620 20:11:14.366746 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 17973
I20260620 20:11:14.372872 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 18109
I20260620 20:11:14.379204 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 17719
I20260620 20:11:14.384694 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:40079
--webserver_interface=127.5.91.190
--webserver_port=37497
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:42053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:40079
--ranger_config_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.91.148:37877/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.91.148:37877/kms with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:11:14.498790 18242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:14.499043 18242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:14.499080 18242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:14.503254 18242 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:11:14.503328 18242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:14.503347 18242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:14.503362 18242 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:11:14.503377 18242 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:11:14.509037 18242 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:42053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.91.148:37877/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:40079
--ranger_config_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:40079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=37497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:14.510012 18242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:14.510998 18242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:14.519171 18250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:14.519150 18247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:14.519266 18248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:14.519490 18242 server_base.cc:1034] running on GCE node
I20260620 20:11:14.520104 18242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:14.521157 18242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:14.522351 18242 hybrid_clock.cc:648] HybridClock initialized: now 1781986274522329 us; error 39 us; skew 500 ppm
Jun 20 20:11:14 dist-test-slave-l8h4 krb5kdc[17273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986274, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:14.525249 18242 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:11:14.526218 18242 webserver.cc:460] Webserver started at http://127.5.91.190:37497/ using document root <none> and password file <none>
I20260620 20:11:14.526772 18242 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:14.526846 18242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 20:11:14 dist-test-slave-l8h4 krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986274, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for HTTP/127.5.91.148@KRBTEST.COM
I20260620 20:11:14.574695 18242 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:11:14.576998 18258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:14.578586 18242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:11:14.578718 18242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "5795af9ec2544ce988385c4671f4cd8e"
format_stamp: "Formatted at 2026-06-20 20:11:12 on dist-test-slave-l8h4"
server_key: "efea7ce3421e27fc1b25c31b378e2548"
server_key_iv: "a36a7098944db97545eafa0f82116e61"
server_key_version: "kuduclusterkey@0"
I20260620 20:11:14.579272 18242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:14.588969 18242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:14.591704 18242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:14.591854 18242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:14.599167 18242 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:40079
I20260620 20:11:14.599258 18310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:40079 every 8 connection(s)
I20260620 20:11:14.600306 18242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:11:14.603179 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 18242
I20260620 20:11:14.603757 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.91.148:37877/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:44283
--local_ip_for_outbound_sockets=127.5.91.129
--tserver_master_addrs=127.5.91.190:40079
--webserver_port=44407
--webserver_interface=127.5.91.129
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:42053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.91.148:37877/kms with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 20:11:14.604867 18311 sys_catalog.cc:263] Verifying existing consensus state
I20260620 20:11:14.607002 18311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e: Bootstrap starting.
I20260620 20:11:14.629891 18311 log.cc:824] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e: Log is configured to *not* fsync() on all Append() calls
I20260620 20:11:14.637918 18311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 20:11:14.638399 18311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e: Bootstrap complete.
I20260620 20:11:14.643767 18311 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5795af9ec2544ce988385c4671f4cd8e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40079 } }
I20260620 20:11:14.644294 18311 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5795af9ec2544ce988385c4671f4cd8e, State: Initialized, Role: FOLLOWER
I20260620 20:11:14.644855 18311 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5795af9ec2544ce988385c4671f4cd8e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40079 } }
I20260620 20:11:14.645013 18311 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:11:14.645083 18311 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:11:14.645177 18311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [term 1 FOLLOWER]: Advancing to term 2
I20260620 20:11:14.647778 18311 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5795af9ec2544ce988385c4671f4cd8e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40079 } }
I20260620 20:11:14.648166 18311 leader_election.cc:302] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5795af9ec2544ce988385c4671f4cd8e; no voters:
I20260620 20:11:14.648507 18311 leader_election.cc:288] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 20:11:14.649020 18315 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [term 2 FOLLOWER]: Leader election won for term 2
I20260620 20:11:14.649361 18315 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [term 2 LEADER]: Becoming Leader. State: Replica: 5795af9ec2544ce988385c4671f4cd8e, State: Running, Role: LEADER
I20260620 20:11:14.649941 18311 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:11:14.649899 18315 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5795af9ec2544ce988385c4671f4cd8e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40079 } }
I20260620 20:11:14.651787 18317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5795af9ec2544ce988385c4671f4cd8e. Latest consensus state: current_term: 2 leader_uuid: "5795af9ec2544ce988385c4671f4cd8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5795af9ec2544ce988385c4671f4cd8e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40079 } } }
I20260620 20:11:14.651966 18317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [sys.catalog]: This master's current role is: LEADER
I20260620 20:11:14.651787 18316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5795af9ec2544ce988385c4671f4cd8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5795af9ec2544ce988385c4671f4cd8e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40079 } } }
I20260620 20:11:14.652122 18316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e [sys.catalog]: This master's current role is: LEADER
I20260620 20:11:14.652305 18311 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 20:11:14.652760 18330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:11:14.659590 18330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:11:14.660276 18330 catalog_manager.cc:1259] Loaded cluster ID: 8921691ee4774ecbbefea84979e2d8ce
I20260620 20:11:14.660334 18330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:11:14.662829 18330 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:11:14.663542 18330 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e: Loaded TSK: 0
I20260620 20:11:14.664846 18330 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 20:11:14.743866 18313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:14.744148 18313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:14.744210 18313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:14.748669 18313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:14.748739 18313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:14.748822 18313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:11:14.755347 18313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:42053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.91.148:37877/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:44283
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=44407
--tserver_master_addrs=127.5.91.190:40079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:14.756816 18313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:14.758229 18313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:14.770274 18350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:14.770166 18349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:14.770166 18352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:14.770749 18313 server_base.cc:1034] running on GCE node
I20260620 20:11:14.771312 18313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:14.771955 18313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:14.773236 18313 hybrid_clock.cc:648] HybridClock initialized: now 1781986274773182 us; error 49 us; skew 500 ppm
Jun 20 20:11:14 dist-test-slave-l8h4 krb5kdc[17273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986274, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:14.776839 18313 init.cc:377] Logged in from keytab as kudu/127.5.91.129@KRBTEST.COM (short username kudu)
I20260620 20:11:14.777969 18313 webserver.cc:460] Webserver started at http://127.5.91.129:44407/ using document root <none> and password file <none>
I20260620 20:11:14.778765 18313 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:14.778928 18313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 20:11:14 dist-test-slave-l8h4 krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986274, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for HTTP/127.5.91.148@KRBTEST.COM
I20260620 20:11:14.799126 18313 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260620 20:11:14.802922 18360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:14.804764 18313 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260620 20:11:14.804962 18313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "8a3519cbf7934273b81427b3d2cd57c5"
format_stamp: "Formatted at 2026-06-20 20:11:13 on dist-test-slave-l8h4"
server_key: "1aeed0c44707342d7b8b27ecaed833f2"
server_key_iv: "d202ffbfd6509a2a170ecae5b88aaeef"
server_key_version: "kuduclusterkey@0"
I20260620 20:11:14.805833 18313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:14.817970 18313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:14.820894 18313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:14.821084 18313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:14.821671 18313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:11:14.822753 18313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:11:14.822815 18313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:14.822873 18313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:11:14.822896 18313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:14.841354 18313 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:44283
I20260620 20:11:14.842916 18473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:44283 every 8 connection(s)
I20260620 20:11:14.844199 18313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:11:14.853336 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 18313
I20260620 20:11:14.853858 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.91.148:37877/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:41615
--local_ip_for_outbound_sockets=127.5.91.130
--tserver_master_addrs=127.5.91.190:40079
--webserver_port=33453
--webserver_interface=127.5.91.130
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:42053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.91.148:37877/kms with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 20:11:14 dist-test-slave-l8h4 krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986274, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:11:14.871279 18476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:14.847499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:40079 (local address 127.5.91.129:60011)
0620 20:11:14.848379 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:14.848439 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:14.849662 (+ 1223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:14.851703 (+ 2041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:14.851713 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:14.852100 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:14.852800 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:14.852822 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:14.855639 (+ 2817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:14.855643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:14.857101 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:14.857114 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:14.858226 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:14.858233 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:14.858439 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:14.859200 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:14.859223 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:14.863061 (+ 3838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:14.867212 (+ 4151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:14.867221 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:14.867240 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:14.867667 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:14.868181 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:14.868184 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:14.868186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:14.868396 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:14.869228 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:14.869235 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:14.869556 (+ 321us) client_negotiation.cc:769] Sending connection context
0620 20:11:14.869767 (+ 211us) client_negotiation.cc:240] Negotiation successful
0620 20:11:14.870129 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":158,"threads_started":1}
I20260620 20:11:14.874186 18477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:14.848917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:60011 (local address 127.5.91.190:40079)
0620 20:11:14.850414 (+ 1497us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:14.850438 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:14.850487 (+ 49us) server_negotiation.cc:407] Connection header received
0620 20:11:14.850793 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:14.850806 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:14.851231 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:14.851607 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:14.853039 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:14.855454 (+ 2415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:14.857261 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:14.857990 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:14.858924 (+ 934us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:14.863488 (+ 4564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:14.863497 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:14.863521 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:14.863572 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:14.867027 (+ 3455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:14.867856 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:14.867864 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:14.867870 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:14.868041 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:14.868539 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:14.868542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:14.868544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:14.869071 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:14.871226 (+ 2155us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:14.872925 (+ 1699us) server_negotiation.cc:299] Negotiation successful
0620 20:11:14.873215 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1305,"thread_start_us":118,"threads_started":1}
I20260620 20:11:14.876956 18474 heartbeater.cc:347] Connected to a master server at 127.5.91.190:40079
I20260620 20:11:14.877198 18474 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:14.877774 18474 heartbeater.cc:511] Master 127.5.91.190:40079 requested a full tablet report, sending...
W20260620 20:11:14.880069 18474 heartbeater.cc:644] Failed to heartbeat to 127.5.91.190:40079 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 20:11:14.880205 18474 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:14.880409 18474 heartbeater.cc:511] Master 127.5.91.190:40079 requested a full tablet report, sending...
I20260620 20:11:14.881289 18474 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:14.881477 18474 heartbeater.cc:511] Master 127.5.91.190:40079 requested a full tablet report, sending...
W20260620 20:11:14.882272 18474 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 20:11:15.027880 18478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:15.028162 18478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:15.028206 18478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:15.032948 18478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:15.033025 18478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:15.033092 18478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:11:15.042021 18478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:42053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.91.148:37877/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:41615
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=33453
--tserver_master_addrs=127.5.91.190:40079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:15.043592 18478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:15.045009 18478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:15.058048 18484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:15.058046 18483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:15.058288 18478 server_base.cc:1034] running on GCE node
W20260620 20:11:15.058310 18486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:15.058816 18478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:15.059614 18478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:15.060829 18478 hybrid_clock.cc:648] HybridClock initialized: now 1781986275060796 us; error 39 us; skew 500 ppm
Jun 20 20:11:15 dist-test-slave-l8h4 krb5kdc[17273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986275, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:15.064754 18478 init.cc:377] Logged in from keytab as kudu/127.5.91.130@KRBTEST.COM (short username kudu)
I20260620 20:11:15.065896 18478 webserver.cc:460] Webserver started at http://127.5.91.130:33453/ using document root <none> and password file <none>
I20260620 20:11:15.066641 18478 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:15.066751 18478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 20:11:15 dist-test-slave-l8h4 krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986275, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for HTTP/127.5.91.148@KRBTEST.COM
I20260620 20:11:15.093263 18478 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 20:11:15.096805 18494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:15.098518 18478 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260620 20:11:15.098667 18478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "2bd4539212d544dfa0f5ae9b90b4b8b7"
format_stamp: "Formatted at 2026-06-20 20:11:13 on dist-test-slave-l8h4"
server_key: "34c03609716b7b890654e450ba308001"
server_key_iv: "e178949b97ba4675b674dd73c231701b"
server_key_version: "kuduclusterkey@0"
I20260620 20:11:15.099447 18478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:15.131920 18478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:15.135672 18478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:15.135885 18478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:15.136655 18478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:11:15.137847 18478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:11:15.137914 18478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:15.137972 18478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:11:15.138005 18478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:15.150786 18478 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:41615
I20260620 20:11:15.152230 18478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:11:15.153090 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 18478
I20260620 20:11:15.153607 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.91.148:37877/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:46361
--local_ip_for_outbound_sockets=127.5.91.131
--tserver_master_addrs=127.5.91.190:40079
--webserver_port=45811
--webserver_interface=127.5.91.131
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:42053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.91.148:37877/kms with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 20:11:15.155328 18608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:41615 every 8 connection(s)
Jun 20 20:11:15 dist-test-slave-l8h4 krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986275, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:11:15.183245 18477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:15.156991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:59979 (local address 127.5.91.190:40079)
0620 20:11:15.158865 (+ 1874us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:15.158869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:15.166831 (+ 7962us) server_negotiation.cc:407] Connection header received
0620 20:11:15.168627 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:15.168633 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:15.168730 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:15.168800 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:15.171084 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:15.171931 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:15.173692 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:15.174068 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:15.174173 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:15.177271 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:15.177276 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:15.177279 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:15.177310 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:15.179613 (+ 2303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:15.180619 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:15.180624 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:15.180626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:15.180744 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:15.181334 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:15.181338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:15.181340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:15.181634 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:15.181724 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:15.182802 (+ 1078us) server_negotiation.cc:299] Negotiation successful
0620 20:11:15.183009 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1760}
I20260620 20:11:15.185838 18612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:15.157426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:40079 (local address 127.5.91.130:59979)
0620 20:11:15.166502 (+ 9076us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:15.166627 (+ 125us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:15.168235 (+ 1608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:15.169586 (+ 1351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:15.169601 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:15.170010 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:15.170818 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:15.170842 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:15.172106 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:15.172110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:15.173542 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:15.173555 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:15.174392 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:15.174395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:15.174622 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:15.175326 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:15.175348 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:15.177042 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:15.179886 (+ 2844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:15.179897 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:15.179928 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:15.180446 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:15.180876 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:15.180879 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:15.180881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:15.181208 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:15.181820 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:15.181829 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:15.182447 (+ 618us) client_negotiation.cc:769] Sending connection context
0620 20:11:15.183985 (+ 1538us) client_negotiation.cc:240] Negotiation successful
0620 20:11:15.184709 (+ 724us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8607,"thread_start_us":478,"threads_started":1}
I20260620 20:11:15.187788 18609 heartbeater.cc:347] Connected to a master server at 127.5.91.190:40079
I20260620 20:11:15.188045 18609 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:15.188638 18609 heartbeater.cc:511] Master 127.5.91.190:40079 requested a full tablet report, sending...
W20260620 20:11:15.190948 18609 heartbeater.cc:644] Failed to heartbeat to 127.5.91.190:40079 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 20:11:15.191078 18609 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:15.191269 18609 heartbeater.cc:511] Master 127.5.91.190:40079 requested a full tablet report, sending...
I20260620 20:11:15.192252 18609 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:15.192438 18609 heartbeater.cc:511] Master 127.5.91.190:40079 requested a full tablet report, sending...
W20260620 20:11:15.193207 18609 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 20:11:15.308297 18611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:15.308650 18611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:15.308722 18611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:15.315573 18611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:15.315670 18611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:15.315765 18611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:11:15.323827 18611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:42053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.91.148:37877/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:46361
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=45811
--tserver_master_addrs=127.5.91.190:40079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:15.325434 18611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:15.326875 18611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:15.340070 18621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:15.340013 18618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:15.340962 18619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:15.341055 18611 server_base.cc:1034] running on GCE node
I20260620 20:11:15.341876 18611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:15.343402 18611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:15.344575 18611 hybrid_clock.cc:648] HybridClock initialized: now 1781986275344544 us; error 49 us; skew 500 ppm
Jun 20 20:11:15 dist-test-slave-l8h4 krb5kdc[17273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986275, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:15.348918 18611 init.cc:377] Logged in from keytab as kudu/127.5.91.131@KRBTEST.COM (short username kudu)
I20260620 20:11:15.350272 18611 webserver.cc:460] Webserver started at http://127.5.91.131:45811/ using document root <none> and password file <none>
I20260620 20:11:15.351063 18611 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:15.351176 18611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 20:11:15 dist-test-slave-l8h4 krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986275, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for HTTP/127.5.91.148@KRBTEST.COM
I20260620 20:11:15.377413 18611 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260620 20:11:15.384239 18629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:15.385718 18611 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20260620 20:11:15.385859 18611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "3b5dacd48f744f4989426a1c1129f244"
format_stamp: "Formatted at 2026-06-20 20:11:14 on dist-test-slave-l8h4"
server_key: "fb171cc3b9aeb7bb54a4df709197f7ea"
server_key_iv: "8019ef22a644c14025bb7287df06f4b0"
server_key_version: "kuduclusterkey@0"
I20260620 20:11:15.386493 18611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:15.399031 18611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:15.403077 18611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:15.403294 18611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:15.404064 18611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:11:15.405150 18611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:11:15.405215 18611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:15.405268 18611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:11:15.405299 18611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:15.416695 18611 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:46361
I20260620 20:11:15.418123 18611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:11:15.419124 18743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:46361 every 8 connection(s)
I20260620 20:11:15.427436 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 18611
Jun 20 20:11:15 dist-test-slave-l8h4 krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986275, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:11:15.445990 18477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:15.421670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:53817 (local address 127.5.91.190:40079)
0620 20:11:15.421895 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:15.421904 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:15.423482 (+ 1578us) server_negotiation.cc:407] Connection header received
0620 20:11:15.424400 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:15.424403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:15.424462 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:15.424536 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:15.426609 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:15.427445 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:15.437895 (+ 10450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:15.438289 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:15.438376 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:15.441832 (+ 3456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:15.441836 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:15.441839 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:15.441868 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:15.443449 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:15.444116 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:15.444120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:15.444122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:15.444208 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:15.444615 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:15.444619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:15.444621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:15.444847 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:15.444927 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:15.445674 (+ 747us) server_negotiation.cc:299] Negotiation successful
0620 20:11:15.445809 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 20:11:15.447705 18746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:15.422422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:40079 (local address 127.5.91.131:53817)
0620 20:11:15.423280 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:15.423325 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:15.424193 (+ 868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:15.424717 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:15.424725 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:15.425209 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:15.426367 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:15.426402 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:15.435867 (+ 9465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:15.435875 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:15.437711 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:15.437730 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:15.438631 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:15.438636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:15.438872 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:15.439600 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:15.439623 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:15.441602 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:15.443590 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:15.443597 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:15.443617 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:15.443981 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:15.444311 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:15.444314 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:15.444316 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:15.444504 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:15.444996 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:15.445004 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:15.445384 (+ 380us) client_negotiation.cc:769] Sending connection context
0620 20:11:15.446279 (+ 895us) client_negotiation.cc:240] Negotiation successful
0620 20:11:15.446652 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":199,"threads_started":1}
I20260620 20:11:15.447989 18748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:15.428134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43284 (local address 127.5.91.190:40079)
0620 20:11:15.436265 (+ 8131us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:15.436271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:15.436289 (+ 18us) server_negotiation.cc:407] Connection header received
0620 20:11:15.436349 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:15.436352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:15.436408 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:15.436510 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:15.437808 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:15.438726 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:15.439994 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:15.440320 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:15.440417 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:15.442051 (+ 1634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:15.442054 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:15.442057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:15.442105 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:15.445641 (+ 3536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:15.446469 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:15.446473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:15.446475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:15.446580 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:15.446902 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:15.446905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:15.446907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:15.447107 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:15.447299 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:15.447714 (+ 415us) server_negotiation.cc:299] Negotiation successful
0620 20:11:15.447839 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8034,"thread_start_us":91,"threads_started":1}
I20260620 20:11:15.448956 18744 heartbeater.cc:347] Connected to a master server at 127.5.91.190:40079
I20260620 20:11:15.449226 18744 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:15.449857 18744 heartbeater.cc:511] Master 127.5.91.190:40079 requested a full tablet report, sending...
W20260620 20:11:15.451674 18744 heartbeater.cc:644] Failed to heartbeat to 127.5.91.190:40079 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 20:11:15.451812 18744 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:15.452008 18744 heartbeater.cc:511] Master 127.5.91.190:40079 requested a full tablet report, sending...
I20260620 20:11:15.452903 18744 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:15.453085 18744 heartbeater.cc:511] Master 127.5.91.190:40079 requested a full tablet report, sending...
W20260620 20:11:15.454218 18744 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 20 20:11:15 dist-test-slave-l8h4 krb5kdc[17273](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781986275, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:15.882525 18474 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:15.882849 18474 heartbeater.cc:511] Master 127.5.91.190:40079 requested a full tablet report, sending...
I20260620 20:11:15.884902 18270 ts_manager.cc:194] Registered new tserver with Master: 8a3519cbf7934273b81427b3d2cd57c5 (127.5.91.129:44283)
I20260620 20:11:15.887583 18270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:60011
I20260620 20:11:16.193450 18609 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:16.193821 18609 heartbeater.cc:511] Master 127.5.91.190:40079 requested a full tablet report, sending...
I20260620 20:11:16.194674 18270 ts_manager.cc:194] Registered new tserver with Master: 2bd4539212d544dfa0f5ae9b90b4b8b7 (127.5.91.130:41615)
I20260620 20:11:16.195245 18270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:59979
Jun 20 20:11:16 dist-test-slave-l8h4 krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986275, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for HTTP/127.5.91.148@KRBTEST.COM
I20260620 20:11:16.454445 18744 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:16.454756 18744 heartbeater.cc:511] Master 127.5.91.190:40079 requested a full tablet report, sending...
I20260620 20:11:16.455509 18270 ts_manager.cc:194] Registered new tserver with Master: 3b5dacd48f744f4989426a1c1129f244 (127.5.91.131:46361)
I20260620 20:11:16.456290 18270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:53817
I20260620 20:11:16.464568 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 20:11:16.484831 18766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:16.466163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43286 (local address 127.5.91.190:40079)
0620 20:11:16.467619 (+ 1456us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:16.467623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:16.470493 (+ 2870us) server_negotiation.cc:407] Connection header received
0620 20:11:16.470679 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:16.470682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:16.470738 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:16.470840 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:16.471678 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.472510 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.473695 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.474031 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.474126 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:16.480633 (+ 6507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:16.480638 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:16.480641 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:16.480680 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:16.482588 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:16.483121 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:16.483125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:16.483127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:16.483225 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:16.483576 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:16.483580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:16.483582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:16.483814 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:16.483915 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:16.484309 (+ 394us) server_negotiation.cc:299] Negotiation successful
0620 20:11:16.484645 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1373,"thread_start_us":91,"threads_started":1}
I20260620 20:11:16.489099 18270 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:11:16.494539 18270 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:11:16.515205 18773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:16.505853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:44283 (local address 127.0.0.1:46340)
0620 20:11:16.507754 (+ 1901us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:16.507803 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:16.507925 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:16.508644 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:16.508651 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:16.508689 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:11:16.508829 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:16.508837 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.511053 (+ 2216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.511056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:16.512382 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:16.512393 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.514617 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.514622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:16.514749 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:16.514789 (+ 40us) client_negotiation.cc:769] Sending connection context
0620 20:11:16.514900 (+ 111us) client_negotiation.cc:240] Negotiation successful
0620 20:11:16.515030 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1737,"thread_start_us":109,"threads_started":1}
I20260620 20:11:16.516099 18774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:16.505956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46340 (local address 127.5.91.129:44283)
0620 20:11:16.508171 (+ 2215us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:16.508177 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:16.508199 (+ 22us) server_negotiation.cc:407] Connection header received
0620 20:11:16.508269 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:16.508273 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:16.508470 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:16.508619 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:11:16.508978 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.510913 (+ 1935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.512525 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.514273 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.514383 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:16.514543 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:16.515649 (+ 1106us) server_negotiation.cc:299] Negotiation successful
0620 20:11:16.515930 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2121,"thread_start_us":95,"threads_started":1}
I20260620 20:11:16.517844 18777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:16.507169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32944 (local address 127.5.91.130:41615)
0620 20:11:16.507498 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:16.507505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:16.509368 (+ 1863us) server_negotiation.cc:407] Connection header received
0620 20:11:16.509570 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:16.509575 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:16.509745 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:16.509850 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:11:16.510634 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.511687 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.516150 (+ 4463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.517007 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.517089 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:16.517223 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:16.517602 (+ 379us) server_negotiation.cc:299] Negotiation successful
0620 20:11:16.517708 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":71,"threads_started":1}
I20260620 20:11:16.518435 18772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:16.507001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:41615 (local address 127.0.0.1:32944)
0620 20:11:16.509280 (+ 2279us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:16.509293 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:16.509431 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:16.509998 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:16.510001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:16.510016 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:11:16.510166 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:16.510173 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.512025 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.512028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:16.513104 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:16.513116 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.517318 (+ 4202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.517321 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:16.517443 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:16.517466 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 20:11:16.518215 (+ 749us) client_negotiation.cc:240] Negotiation successful
0620 20:11:16.518299 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2222,"thread_start_us":82,"threads_started":1}
I20260620 20:11:16.520260 18403 tablet_service.cc:1467] Processing CreateTablet for tablet 27f53d77e6184c039d5d17a1cb8a8f2d (DEFAULT_TABLE table=test-table [id=0236e1d5d0a842aaa2d6b969491616ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:11:16.520089 18776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:16.505924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:46361 (local address 127.0.0.1:49766)
0620 20:11:16.507783 (+ 1859us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:16.507822 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:16.507926 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:16.511795 (+ 3869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:16.511798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:16.511818 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:11:16.511907 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:16.511912 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.514673 (+ 2761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.514677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:16.515988 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:16.515999 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.519736 (+ 3737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.519739 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:16.519825 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:16.519847 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 20:11:16.519909 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 20:11:16.519973 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1793,"thread_start_us":90,"threads_started":1}
I20260620 20:11:16.521397 18775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:16.506074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49766 (local address 127.5.91.131:46361)
0620 20:11:16.508901 (+ 2827us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:16.508907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:16.508928 (+ 21us) server_negotiation.cc:407] Connection header received
0620 20:11:16.508997 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:16.509001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:16.510336 (+ 1335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:16.510464 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:11:16.513259 (+ 2795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.514529 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.518666 (+ 4137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.519454 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.519554 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:16.519677 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:16.521149 (+ 1472us) server_negotiation.cc:299] Negotiation successful
0620 20:11:16.521250 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2730,"thread_start_us":113,"threads_started":1}
I20260620 20:11:16.522686 18403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27f53d77e6184c039d5d17a1cb8a8f2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:11:16.524215 18678 tablet_service.cc:1467] Processing CreateTablet for tablet 27f53d77e6184c039d5d17a1cb8a8f2d (DEFAULT_TABLE table=test-table [id=0236e1d5d0a842aaa2d6b969491616ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:11:16.524219 18528 tablet_service.cc:1467] Processing CreateTablet for tablet 27f53d77e6184c039d5d17a1cb8a8f2d (DEFAULT_TABLE table=test-table [id=0236e1d5d0a842aaa2d6b969491616ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:11:16.525187 18528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27f53d77e6184c039d5d17a1cb8a8f2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:11:16.525197 18678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27f53d77e6184c039d5d17a1cb8a8f2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:11:16.534152 18778 tablet_bootstrap.cc:492] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244: Bootstrap starting.
I20260620 20:11:16.534494 18780 tablet_bootstrap.cc:492] T 27f53d77e6184c039d5d17a1cb8a8f2d P 2bd4539212d544dfa0f5ae9b90b4b8b7: Bootstrap starting.
I20260620 20:11:16.536793 18780 tablet_bootstrap.cc:654] T 27f53d77e6184c039d5d17a1cb8a8f2d P 2bd4539212d544dfa0f5ae9b90b4b8b7: Neither blocks nor log segments found. Creating new log.
I20260620 20:11:16.537606 18778 tablet_bootstrap.cc:654] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244: Neither blocks nor log segments found. Creating new log.
I20260620 20:11:16.537655 18780 log.cc:824] T 27f53d77e6184c039d5d17a1cb8a8f2d P 2bd4539212d544dfa0f5ae9b90b4b8b7: Log is configured to *not* fsync() on all Append() calls
I20260620 20:11:16.538398 18779 tablet_bootstrap.cc:492] T 27f53d77e6184c039d5d17a1cb8a8f2d P 8a3519cbf7934273b81427b3d2cd57c5: Bootstrap starting.
I20260620 20:11:16.538591 18778 log.cc:824] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244: Log is configured to *not* fsync() on all Append() calls
I20260620 20:11:16.540985 18779 tablet_bootstrap.cc:654] T 27f53d77e6184c039d5d17a1cb8a8f2d P 8a3519cbf7934273b81427b3d2cd57c5: Neither blocks nor log segments found. Creating new log.
I20260620 20:11:16.541491 18778 tablet_bootstrap.cc:492] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244: No bootstrap required, opened a new log
I20260620 20:11:16.541692 18778 ts_tablet_manager.cc:1397] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260620 20:11:16.541934 18779 log.cc:824] T 27f53d77e6184c039d5d17a1cb8a8f2d P 8a3519cbf7934273b81427b3d2cd57c5: Log is configured to *not* fsync() on all Append() calls
I20260620 20:11:16.544086 18779 tablet_bootstrap.cc:492] T 27f53d77e6184c039d5d17a1cb8a8f2d P 8a3519cbf7934273b81427b3d2cd57c5: No bootstrap required, opened a new log
I20260620 20:11:16.544274 18779 ts_tablet_manager.cc:1397] T 27f53d77e6184c039d5d17a1cb8a8f2d P 8a3519cbf7934273b81427b3d2cd57c5: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260620 20:11:16.546190 18778 raft_consensus.cc:348] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a3519cbf7934273b81427b3d2cd57c5" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44283 } } peers { permanent_uuid: "3b5dacd48f744f4989426a1c1129f244" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46361 } } peers { permanent_uuid: "2bd4539212d544dfa0f5ae9b90b4b8b7" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41615 } }
I20260620 20:11:16.546478 18778 raft_consensus.cc:374] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:11:16.546531 18778 raft_consensus.cc:729] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b5dacd48f744f4989426a1c1129f244, State: Initialized, Role: FOLLOWER
I20260620 20:11:16.548123 18778 consensus_queue.cc:260] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a3519cbf7934273b81427b3d2cd57c5" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44283 } } peers { permanent_uuid: "3b5dacd48f744f4989426a1c1129f244" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46361 } } peers { permanent_uuid: "2bd4539212d544dfa0f5ae9b90b4b8b7" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41615 } }
I20260620 20:11:16.548400 18779 raft_consensus.cc:348] T 27f53d77e6184c039d5d17a1cb8a8f2d P 8a3519cbf7934273b81427b3d2cd57c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a3519cbf7934273b81427b3d2cd57c5" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44283 } } peers { permanent_uuid: "3b5dacd48f744f4989426a1c1129f244" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46361 } } peers { permanent_uuid: "2bd4539212d544dfa0f5ae9b90b4b8b7" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41615 } }
I20260620 20:11:16.548650 18779 raft_consensus.cc:374] T 27f53d77e6184c039d5d17a1cb8a8f2d P 8a3519cbf7934273b81427b3d2cd57c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:11:16.548709 18779 raft_consensus.cc:729] T 27f53d77e6184c039d5d17a1cb8a8f2d P 8a3519cbf7934273b81427b3d2cd57c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a3519cbf7934273b81427b3d2cd57c5, State: Initialized, Role: FOLLOWER
I20260620 20:11:16.549268 18778 ts_tablet_manager.cc:1428] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260620 20:11:16.549679 18744 heartbeater.cc:503] Master 127.5.91.190:40079 was elected leader, sending a full tablet report...
I20260620 20:11:16.551681 18780 tablet_bootstrap.cc:492] T 27f53d77e6184c039d5d17a1cb8a8f2d P 2bd4539212d544dfa0f5ae9b90b4b8b7: No bootstrap required, opened a new log
I20260620 20:11:16.551867 18780 ts_tablet_manager.cc:1397] T 27f53d77e6184c039d5d17a1cb8a8f2d P 2bd4539212d544dfa0f5ae9b90b4b8b7: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.000s
I20260620 20:11:16.556068 18780 raft_consensus.cc:348] T 27f53d77e6184c039d5d17a1cb8a8f2d P 2bd4539212d544dfa0f5ae9b90b4b8b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a3519cbf7934273b81427b3d2cd57c5" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44283 } } peers { permanent_uuid: "3b5dacd48f744f4989426a1c1129f244" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46361 } } peers { permanent_uuid: "2bd4539212d544dfa0f5ae9b90b4b8b7" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41615 } }
I20260620 20:11:16.556382 18780 raft_consensus.cc:374] T 27f53d77e6184c039d5d17a1cb8a8f2d P 2bd4539212d544dfa0f5ae9b90b4b8b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:11:16.556438 18780 raft_consensus.cc:729] T 27f53d77e6184c039d5d17a1cb8a8f2d P 2bd4539212d544dfa0f5ae9b90b4b8b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bd4539212d544dfa0f5ae9b90b4b8b7, State: Initialized, Role: FOLLOWER
I20260620 20:11:16.549459 18779 consensus_queue.cc:260] T 27f53d77e6184c039d5d17a1cb8a8f2d P 8a3519cbf7934273b81427b3d2cd57c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a3519cbf7934273b81427b3d2cd57c5" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44283 } } peers { permanent_uuid: "3b5dacd48f744f4989426a1c1129f244" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46361 } } peers { permanent_uuid: "2bd4539212d544dfa0f5ae9b90b4b8b7" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41615 } }
I20260620 20:11:16.556896 18780 consensus_queue.cc:260] T 27f53d77e6184c039d5d17a1cb8a8f2d P 2bd4539212d544dfa0f5ae9b90b4b8b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a3519cbf7934273b81427b3d2cd57c5" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44283 } } peers { permanent_uuid: "3b5dacd48f744f4989426a1c1129f244" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46361 } } peers { permanent_uuid: "2bd4539212d544dfa0f5ae9b90b4b8b7" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41615 } }
I20260620 20:11:16.557796 18779 ts_tablet_manager.cc:1428] T 27f53d77e6184c039d5d17a1cb8a8f2d P 8a3519cbf7934273b81427b3d2cd57c5: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20260620 20:11:16.558533 18780 ts_tablet_manager.cc:1428] T 27f53d77e6184c039d5d17a1cb8a8f2d P 2bd4539212d544dfa0f5ae9b90b4b8b7: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260620 20:11:16.558770 18474 heartbeater.cc:503] Master 127.5.91.190:40079 was elected leader, sending a full tablet report...
I20260620 20:11:16.558866 18609 heartbeater.cc:503] Master 127.5.91.190:40079 was elected leader, sending a full tablet report...
W20260620 20:11:16.601928 18475 tablet.cc:2369] T 27f53d77e6184c039d5d17a1cb8a8f2d P 8a3519cbf7934273b81427b3d2cd57c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 20 20:11:16 dist-test-slave-l8h4 krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986275, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for HTTP/127.5.91.148@KRBTEST.COM
W20260620 20:11:16.666369 18610 tablet.cc:2369] T 27f53d77e6184c039d5d17a1cb8a8f2d P 2bd4539212d544dfa0f5ae9b90b4b8b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 20:11:16.671104 18745 tablet.cc:2369] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:11:16.757833 18784 raft_consensus.cc:482] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:11:16.758088 18784 raft_consensus.cc:504] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a3519cbf7934273b81427b3d2cd57c5" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44283 } } peers { permanent_uuid: "3b5dacd48f744f4989426a1c1129f244" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46361 } } peers { permanent_uuid: "2bd4539212d544dfa0f5ae9b90b4b8b7" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41615 } }
I20260620 20:11:16.759493 18784 leader_election.cc:288] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a3519cbf7934273b81427b3d2cd57c5 (127.5.91.129:44283), 2bd4539212d544dfa0f5ae9b90b4b8b7 (127.5.91.130:41615)
I20260620 20:11:16.765842 18777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:16.760475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:39717 (local address 127.5.91.130:41615)
0620 20:11:16.760619 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:16.760623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:16.761283 (+ 660us) server_negotiation.cc:407] Connection header received
0620 20:11:16.761512 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:16.761516 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:16.761570 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:16.761617 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:11:16.762093 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.762889 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.764502 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.765095 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.765195 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:16.765250 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:16.765610 (+ 360us) server_negotiation.cc:299] Negotiation successful
0620 20:11:16.765691 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 20:11:16.765861 18774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:16.760982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:53553 (local address 127.5.91.129:44283)
0620 20:11:16.761098 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:16.761102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:16.761345 (+ 243us) server_negotiation.cc:407] Connection header received
0620 20:11:16.761546 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:16.761549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:16.761600 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:16.761644 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:11:16.762119 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.762909 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.764553 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.765141 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.765250 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:16.765300 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:16.765651 (+ 351us) server_negotiation.cc:299] Negotiation successful
0620 20:11:16.765734 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 20:11:16.766400 18790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:16.759945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:41615 (local address 127.5.91.131:39717)
0620 20:11:16.761188 (+ 1243us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:16.761203 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:16.761372 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:16.761769 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:16.761772 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:16.761789 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:11:16.761964 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:16.761972 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.763031 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.763034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:16.764346 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:16.764357 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.765347 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.765350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:16.765467 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:16.765491 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 20:11:16.766123 (+ 632us) client_negotiation.cc:240] Negotiation successful
0620 20:11:16.766228 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1175,"mutex_wait_us":22,"spinlock_wait_cycles":768,"thread_start_us":108,"threads_started":1}
I20260620 20:11:16.766707 18789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:16.760844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:44283 (local address 127.5.91.131:53553)
0620 20:11:16.761259 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:16.761271 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:16.761426 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:16.761773 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:16.761776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:16.761862 (+ 86us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:11:16.761990 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:16.761997 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.763045 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.763047 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:16.764419 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:16.764430 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.765415 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.765418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:16.765520 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:16.765540 (+ 20us) client_negotiation.cc:769] Sending connection context
0620 20:11:16.766314 (+ 774us) client_negotiation.cc:240] Negotiation successful
0620 20:11:16.766405 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":53,"spinlock_wait_cycles":43648,"thread_start_us":100,"threads_started":1}
I20260620 20:11:16.767093 18544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27f53d77e6184c039d5d17a1cb8a8f2d" candidate_uuid: "3b5dacd48f744f4989426a1c1129f244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bd4539212d544dfa0f5ae9b90b4b8b7" is_pre_election: true
I20260620 20:11:16.767443 18544 raft_consensus.cc:2454] T 27f53d77e6184c039d5d17a1cb8a8f2d P 2bd4539212d544dfa0f5ae9b90b4b8b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b5dacd48f744f4989426a1c1129f244 in term 0.
I20260620 20:11:16.767951 18428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27f53d77e6184c039d5d17a1cb8a8f2d" candidate_uuid: "3b5dacd48f744f4989426a1c1129f244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a3519cbf7934273b81427b3d2cd57c5" is_pre_election: true
I20260620 20:11:16.768101 18631 leader_election.cc:302] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bd4539212d544dfa0f5ae9b90b4b8b7, 3b5dacd48f744f4989426a1c1129f244; no voters:
I20260620 20:11:16.768222 18428 raft_consensus.cc:2454] T 27f53d77e6184c039d5d17a1cb8a8f2d P 8a3519cbf7934273b81427b3d2cd57c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b5dacd48f744f4989426a1c1129f244 in term 0.
I20260620 20:11:16.768446 18784 raft_consensus.cc:2781] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:11:16.768539 18784 raft_consensus.cc:482] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:11:16.768587 18784 raft_consensus.cc:3037] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:11:16.769907 18784 raft_consensus.cc:504] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a3519cbf7934273b81427b3d2cd57c5" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44283 } } peers { permanent_uuid: "3b5dacd48f744f4989426a1c1129f244" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46361 } } peers { permanent_uuid: "2bd4539212d544dfa0f5ae9b90b4b8b7" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41615 } }
I20260620 20:11:16.770323 18784 leader_election.cc:288] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244 [CANDIDATE]: Term 1 election: Requested vote from peers 8a3519cbf7934273b81427b3d2cd57c5 (127.5.91.129:44283), 2bd4539212d544dfa0f5ae9b90b4b8b7 (127.5.91.130:41615)
I20260620 20:11:16.771430 18428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27f53d77e6184c039d5d17a1cb8a8f2d" candidate_uuid: "3b5dacd48f744f4989426a1c1129f244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a3519cbf7934273b81427b3d2cd57c5"
I20260620 20:11:16.771664 18428 raft_consensus.cc:3037] T 27f53d77e6184c039d5d17a1cb8a8f2d P 8a3519cbf7934273b81427b3d2cd57c5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:11:16.772980 18428 raft_consensus.cc:2454] T 27f53d77e6184c039d5d17a1cb8a8f2d P 8a3519cbf7934273b81427b3d2cd57c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b5dacd48f744f4989426a1c1129f244 in term 1.
I20260620 20:11:16.773342 18633 leader_election.cc:302] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b5dacd48f744f4989426a1c1129f244, 8a3519cbf7934273b81427b3d2cd57c5; no voters:
I20260620 20:11:16.773947 18544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27f53d77e6184c039d5d17a1cb8a8f2d" candidate_uuid: "3b5dacd48f744f4989426a1c1129f244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bd4539212d544dfa0f5ae9b90b4b8b7"
I20260620 20:11:16.774121 18784 raft_consensus.cc:2781] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:11:16.774207 18544 raft_consensus.cc:3037] T 27f53d77e6184c039d5d17a1cb8a8f2d P 2bd4539212d544dfa0f5ae9b90b4b8b7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:11:16.774372 18784 raft_consensus.cc:686] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244 [term 1 LEADER]: Becoming Leader. State: Replica: 3b5dacd48f744f4989426a1c1129f244, State: Running, Role: LEADER
I20260620 20:11:16.774722 18784 consensus_queue.cc:237] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a3519cbf7934273b81427b3d2cd57c5" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44283 } } peers { permanent_uuid: "3b5dacd48f744f4989426a1c1129f244" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46361 } } peers { permanent_uuid: "2bd4539212d544dfa0f5ae9b90b4b8b7" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41615 } }
I20260620 20:11:16.775483 18544 raft_consensus.cc:2454] T 27f53d77e6184c039d5d17a1cb8a8f2d P 2bd4539212d544dfa0f5ae9b90b4b8b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b5dacd48f744f4989426a1c1129f244 in term 1.
I20260620 20:11:16.777854 18270 catalog_manager.cc:5495] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b5dacd48f744f4989426a1c1129f244 (127.5.91.131). New cstate: current_term: 1 leader_uuid: "3b5dacd48f744f4989426a1c1129f244" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a3519cbf7934273b81427b3d2cd57c5" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b5dacd48f744f4989426a1c1129f244" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bd4539212d544dfa0f5ae9b90b4b8b7" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41615 } health_report { overall_health: UNKNOWN } } }
I20260620 20:11:16.811712 18775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:16.806418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49768 (local address 127.5.91.131:46361)
0620 20:11:16.806568 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:16.806583 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:16.806632 (+ 49us) server_negotiation.cc:407] Connection header received
0620 20:11:16.806820 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:16.806823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:16.806899 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:16.807015 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:11:16.807412 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.808227 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.809494 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:16.809808 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:16.809909 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:16.810361 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:11:16.810981 (+ 620us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:11:16.811102 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:16.811470 (+ 368us) server_negotiation.cc:299] Negotiation successful
0620 20:11:16.811533 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 20:11:16.819422 18544 raft_consensus.cc:1261] T 27f53d77e6184c039d5d17a1cb8a8f2d P 2bd4539212d544dfa0f5ae9b90b4b8b7 [term 1 FOLLOWER]: Refusing update from remote peer 3b5dacd48f744f4989426a1c1129f244: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:11:16.820508 18428 raft_consensus.cc:1261] T 27f53d77e6184c039d5d17a1cb8a8f2d P 8a3519cbf7934273b81427b3d2cd57c5 [term 1 FOLLOWER]: Refusing update from remote peer 3b5dacd48f744f4989426a1c1129f244: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:11:16.821154 18791 consensus_queue.cc:1035] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bd4539212d544dfa0f5ae9b90b4b8b7" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 41615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:11:16.821328 18784 consensus_queue.cc:1035] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a3519cbf7934273b81427b3d2cd57c5" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 44283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:11:16.830467 18794 mvcc.cc:205] Tried to move back new op lower bound from 7299015789842313216 to 7299015789672083456. Current Snapshot: MvccSnapshot[applied={T|T < 7299015789842313216}]
I20260620 20:11:16.830657 18795 mvcc.cc:205] Tried to move back new op lower bound from 7299015789842313216 to 7299015789672083456. Current Snapshot: MvccSnapshot[applied={T|T < 7299015789842313216}]
I20260620 20:11:16.842335 18796 mvcc.cc:205] Tried to move back new op lower bound from 7299015789842313216 to 7299015789672083456. Current Snapshot: MvccSnapshot[applied={T|T < 7299015789842313216}]
I20260620 20:11:16.843019 18271 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:11:16.843199 18271 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:11:16.845949 18271 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5795af9ec2544ce988385c4671f4cd8e: Sending DeleteTablet for 3 replicas of tablet 27f53d77e6184c039d5d17a1cb8a8f2d
I20260620 20:11:16.846815 18403 tablet_service.cc:1514] Processing DeleteTablet for tablet 27f53d77e6184c039d5d17a1cb8a8f2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:11:16 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:46340
I20260620 20:11:16.847178 18678 tablet_service.cc:1514] Processing DeleteTablet for tablet 27f53d77e6184c039d5d17a1cb8a8f2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:11:16 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:49766
I20260620 20:11:16.847744 18803 tablet_replica.cc:330] T 27f53d77e6184c039d5d17a1cb8a8f2d P 8a3519cbf7934273b81427b3d2cd57c5: stopping tablet replica
I20260620 20:11:16.848079 18803 raft_consensus.cc:2229] T 27f53d77e6184c039d5d17a1cb8a8f2d P 8a3519cbf7934273b81427b3d2cd57c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:11:16.848433 18803 raft_consensus.cc:2258] T 27f53d77e6184c039d5d17a1cb8a8f2d P 8a3519cbf7934273b81427b3d2cd57c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:11:16.849110 18528 tablet_service.cc:1514] Processing DeleteTablet for tablet 27f53d77e6184c039d5d17a1cb8a8f2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:11:16 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:32944
I20260620 20:11:16.850183 18803 ts_tablet_manager.cc:1905] T 27f53d77e6184c039d5d17a1cb8a8f2d P 8a3519cbf7934273b81427b3d2cd57c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:11:16.853418 18803 ts_tablet_manager.cc:1918] T 27f53d77e6184c039d5d17a1cb8a8f2d P 8a3519cbf7934273b81427b3d2cd57c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:11:16.853538 18803 log.cc:1196] T 27f53d77e6184c039d5d17a1cb8a8f2d P 8a3519cbf7934273b81427b3d2cd57c5: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-0/wal/wals/27f53d77e6184c039d5d17a1cb8a8f2d
I20260620 20:11:16.853888 18803 ts_tablet_manager.cc:1939] T 27f53d77e6184c039d5d17a1cb8a8f2d P 8a3519cbf7934273b81427b3d2cd57c5: Deleting consensus metadata
I20260620 20:11:16.856001 18262 catalog_manager.cc:4841] TS 8a3519cbf7934273b81427b3d2cd57c5 (127.5.91.129:44283): tablet 27f53d77e6184c039d5d17a1cb8a8f2d (table test-table [id=0236e1d5d0a842aaa2d6b969491616ed]) successfully deleted
I20260620 20:11:16.851591 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 18313
I20260620 20:11:16.858567 18805 tablet_replica.cc:330] T 27f53d77e6184c039d5d17a1cb8a8f2d P 2bd4539212d544dfa0f5ae9b90b4b8b7: stopping tablet replica
I20260620 20:11:16.859145 18805 raft_consensus.cc:2229] T 27f53d77e6184c039d5d17a1cb8a8f2d P 2bd4539212d544dfa0f5ae9b90b4b8b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:11:16.859618 18805 raft_consensus.cc:2258] T 27f53d77e6184c039d5d17a1cb8a8f2d P 2bd4539212d544dfa0f5ae9b90b4b8b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:11:16.860472 18804 tablet_replica.cc:330] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244: stopping tablet replica
I20260620 20:11:16.860859 18804 raft_consensus.cc:2229] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244 [term 1 LEADER]: Raft consensus shutting down.
I20260620 20:11:16.861563 18804 raft_consensus.cc:2258] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:11:16.863687 18804 ts_tablet_manager.cc:1905] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:11:16.866077 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 18478
I20260620 20:11:16.867214 18804 ts_tablet_manager.cc:1918] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:11:16.867324 18804 log.cc:1196] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/27f53d77e6184c039d5d17a1cb8a8f2d
I20260620 20:11:16.867642 18804 ts_tablet_manager.cc:1939] T 27f53d77e6184c039d5d17a1cb8a8f2d P 3b5dacd48f744f4989426a1c1129f244: Deleting consensus metadata
I20260620 20:11:16.868974 18260 catalog_manager.cc:4841] TS 3b5dacd48f744f4989426a1c1129f244 (127.5.91.131:46361): tablet 27f53d77e6184c039d5d17a1cb8a8f2d (table test-table [id=0236e1d5d0a842aaa2d6b969491616ed]) successfully deleted
W20260620 20:11:16.874123 18260 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.91.130:41615 (error 108)
I20260620 20:11:16.875043 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 18611
I20260620 20:11:16.878463 18773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:16.878007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:41615 (local address 127.0.0.1:32954)
0620 20:11:16.878290 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:16.878364 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.130:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193}
W20260620 20:11:16.878787 18260 catalog_manager.cc:4584] TS 2bd4539212d544dfa0f5ae9b90b4b8b7 (127.5.91.130:41615): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 27f53d77e6184c039d5d17a1cb8a8f2d: Network error: Client connection negotiation failed: client connection to 127.5.91.130:41615: connect: Connection refused (error 111)
I20260620 20:11:16.882441 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 18242
I20260620 20:11:16.888242 5486 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260620 20:11:17.223739 5486 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260620 20:11:17.223850 5486 mini_ranger.cc:66] Stopping Ranger...
I20260620 20:11:17.559340 5486 mini_ranger.cc:68] Stopped Ranger
2026-06-20 20:11:17.559 UTC [17303] LOG: received smart shutdown request
2026-06-20 20:11:17.563 UTC [17303] LOG: background worker "logical replication launcher" (PID 17312) exited with exit code 1
2026-06-20 20:11:17.564 UTC [17307] LOG: shutting down
2026-06-20 20:11:17.751 UTC [17303] LOG: database system is shut down
2026-06-20T20:11:17Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52372 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:11:17 dist-test-slave-l8h4 krb5kdc[18817](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:11:17 dist-test-slave-l8h4 krb5kdc[18817](info): set up 2 sockets
Jun 20 20:11:17 dist-test-slave-l8h4 krb5kdc[18817](info): commencing operation
krb5kdc: starting...
W20260620 20:11:18.814831 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.949s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:11:18 dist-test-slave-l8h4 krb5kdc[18817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:11:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:11:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:11:18.937062 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:40449
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:42229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:40449,127.5.91.189:44473,127.5.91.188:34493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:11:19.058007 18830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:19.058343 18830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:19.058385 18830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:19.062467 18830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:11:19.062528 18830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:19.062546 18830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:19.062562 18830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:11:19.062577 18830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:11:19.069159 18830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:42229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:40449,127.5.91.189:44473,127.5.91.188:34493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:40449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:19.070370 18830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:19.071383 18830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:19.079878 18838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:19.079789 18835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:19.080001 18836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:19.080101 18830 server_base.cc:1034] running on GCE node
I20260620 20:11:19.080711 18830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:19.081862 18830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:19.083052 18830 hybrid_clock.cc:648] HybridClock initialized: now 1781986279083050 us; error 52 us; skew 500 ppm
Jun 20 20:11:19 dist-test-slave-l8h4 krb5kdc[18817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986279, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:19.086014 18830 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:11:19.087088 18830 webserver.cc:460] Webserver started at http://127.5.91.190:45317/ using document root <none> and password file <none>
I20260620 20:11:19.087677 18830 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:19.087745 18830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:19.087941 18830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:11:19.090000 18830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "8393b8ea6c40403ea1afb28622f595c9"
format_stamp: "Formatted at 2026-06-20 20:11:19 on dist-test-slave-l8h4"
server_key: "7214189cf63167811cdb76acfa093e78"
server_key_iv: "16547a753242196ca30446353e277b7b"
server_key_version: "encryptionkey@0"
I20260620 20:11:19.090622 18830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "8393b8ea6c40403ea1afb28622f595c9"
format_stamp: "Formatted at 2026-06-20 20:11:19 on dist-test-slave-l8h4"
server_key: "7214189cf63167811cdb76acfa093e78"
server_key_iv: "16547a753242196ca30446353e277b7b"
server_key_version: "encryptionkey@0"
I20260620 20:11:19.094444 18830 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260620 20:11:19.097013 18845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:19.098223 18830 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:11:19.098340 18830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "8393b8ea6c40403ea1afb28622f595c9"
format_stamp: "Formatted at 2026-06-20 20:11:19 on dist-test-slave-l8h4"
server_key: "7214189cf63167811cdb76acfa093e78"
server_key_iv: "16547a753242196ca30446353e277b7b"
server_key_version: "encryptionkey@0"
I20260620 20:11:19.098440 18830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:19.114104 18830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:19.117678 18830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:19.117877 18830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:19.125278 18830 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:40449
I20260620 20:11:19.125300 18897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:40449 every 8 connection(s)
I20260620 20:11:19.126263 18830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:11:19.128955 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 18830
I20260620 20:11:19.129086 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:11:19.129099 18898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:11:19.129396 5486 external_mini_cluster.cc:1428] Setting key 583e32b6dc1b4dab36f15c86d0231452
I20260620 20:11:19.133571 18898 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40449 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.91.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.189@KRBTEST.COM" created.
Jun 20 20:11:19 dist-test-slave-l8h4 krb5kdc[18817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986279, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:11:19.149120 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.136105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:40449 (local address 127.0.0.1:38822)
0620 20:11:19.136651 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.136685 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:19.137550 (+ 865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:19.138189 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:19.138193 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:19.138599 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:19.138883 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:19.138897 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.140298 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.140301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:19.141070 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:19.141080 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.141915 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.141920 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:19.142041 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.142621 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:19.142640 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:19.144453 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:19.146194 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:19.146203 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:19.146214 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:19.146529 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:19.146890 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:19.146893 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:19.146895 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:19.147032 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:19.147698 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:19.147706 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:19.147833 (+ 127us) client_negotiation.cc:769] Sending connection context
0620 20:11:19.147966 (+ 133us) client_negotiation.cc:240] Negotiation successful
0620 20:11:19.148349 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":118,"thread_start_us":151,"threads_started":1}
I20260620 20:11:19.149134 18901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.136104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38822 (local address 127.5.91.190:40449)
0620 20:11:19.136525 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:19.136542 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:19.136782 (+ 240us) server_negotiation.cc:407] Connection header received
0620 20:11:19.137822 (+ 1040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:19.137827 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:19.138006 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:19.138186 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:19.139019 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.140156 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.141188 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.141740 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.141877 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.144597 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:19.144603 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:19.144613 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:19.144634 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:19.146064 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.146649 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.146654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.146655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.146786 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.147172 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.147176 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.147177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.147567 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:19.147696 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:19.148055 (+ 359us) server_negotiation.cc:299] Negotiation successful
0620 20:11:19.148266 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":279,"thread_start_us":117,"threads_started":1}
I20260620 20:11:19.151738 18898 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } has no permanent_uuid. Determining permanent_uuid...
I20260620 20:11:19.152532 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.152262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:44473 (local address 127.0.0.1:47556)
0620 20:11:19.152397 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.152465 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.189:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260620 20:11:19.152823 18847 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.91.189:44473: connect: Connection refused (error 111)
I20260620 20:11:19.153697 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.153448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:44473 (local address 127.0.0.1:47570)
0620 20:11:19.153599 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.153646 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.189:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260620 20:11:19.153855 18898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.189:44473: Network error: Client connection negotiation failed: client connection to 127.5.91.189:44473: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.189.keytab.
Entry for principal kudu/127.5.91.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.189.keytab.
I20260620 20:11:19.157147 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.189:44473
--webserver_interface=127.5.91.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.189.keytab
--principal=kudu/127.5.91.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:42229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:40449,127.5.91.189:44473,127.5.91.188:34493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 20:11:19.203205 18898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } attempt: 1
I20260620 20:11:19.204149 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.203833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:44473 (local address 127.0.0.1:47582)
0620 20:11:19.203997 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.204065 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.189:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260620 20:11:19.205018 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.204708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:44473 (local address 127.0.0.1:47590)
0620 20:11:19.204907 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.204958 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.189:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260620 20:11:19.205139 18898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.189:44473: Network error: Client connection negotiation failed: client connection to 127.5.91.189:44473: connect: Connection refused (error 111)
W20260620 20:11:19.269126 18906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:19.269407 18906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:19.269447 18906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260620 20:11:19.273311 18898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } attempt: 2
W20260620 20:11:19.273748 18906 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:11:19.273808 18906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:19.273828 18906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:19.273844 18906 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:11:19.273869 18906 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:11:19.274174 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.273848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:44473 (local address 127.0.0.1:47600)
0620 20:11:19.274008 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.274071 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.189:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260620 20:11:19.274806 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.274607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:44473 (local address 127.0.0.1:47604)
0620 20:11:19.274705 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.274751 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.189:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 20:11:19.274923 18898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.189:44473: Network error: Client connection negotiation failed: client connection to 127.5.91.189:44473: connect: Connection refused (error 111)
I20260620 20:11:19.279634 18906 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:42229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:40449,127.5.91.189:44473,127.5.91.188:34493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.189.keytab
--principal=kudu/127.5.91.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.189:44473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.91.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:19.280709 18906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:19.281684 18906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:19.290696 18912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:19.290709 18914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:19.290781 18911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:19.290824 18906 server_base.cc:1034] running on GCE node
I20260620 20:11:19.291913 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:19.293102 18906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:19.294241 18906 hybrid_clock.cc:648] HybridClock initialized: now 1781986279294220 us; error 36 us; skew 500 ppm
Jun 20 20:11:19 dist-test-slave-l8h4 krb5kdc[18817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986279, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:19.297885 18906 init.cc:377] Logged in from keytab as kudu/127.5.91.189@KRBTEST.COM (short username kudu)
I20260620 20:11:19.298897 18906 webserver.cc:460] Webserver started at http://127.5.91.189:46637/ using document root <none> and password file <none>
I20260620 20:11:19.299554 18906 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:19.299638 18906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:19.299834 18906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:11:19.301854 18906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-1/data/instance:
uuid: "8b2c551cde6f4d37879989259c70a69c"
format_stamp: "Formatted at 2026-06-20 20:11:19 on dist-test-slave-l8h4"
server_key: "36ba3fb21593e5f61df02a60774b6bd5"
server_key_iv: "191df1a4801dbfa6c2477872e707ca0a"
server_key_version: "encryptionkey@0"
I20260620 20:11:19.302397 18906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-1/wal/instance:
uuid: "8b2c551cde6f4d37879989259c70a69c"
format_stamp: "Formatted at 2026-06-20 20:11:19 on dist-test-slave-l8h4"
server_key: "36ba3fb21593e5f61df02a60774b6bd5"
server_key_iv: "191df1a4801dbfa6c2477872e707ca0a"
server_key_version: "encryptionkey@0"
I20260620 20:11:19.306070 18906 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260620 20:11:19.308547 18921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:19.309943 18906 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260620 20:11:19.310068 18906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-1/wal
uuid: "8b2c551cde6f4d37879989259c70a69c"
format_stamp: "Formatted at 2026-06-20 20:11:19 on dist-test-slave-l8h4"
server_key: "36ba3fb21593e5f61df02a60774b6bd5"
server_key_iv: "191df1a4801dbfa6c2477872e707ca0a"
server_key_version: "encryptionkey@0"
I20260620 20:11:19.310227 18906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:19.330380 18906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:19.333784 18906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:19.333981 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:19.341845 18906 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.189:44473
I20260620 20:11:19.341894 18973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.189:44473 every 8 connection(s)
I20260620 20:11:19.342880 18906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-1/data/info.pb
I20260620 20:11:19.344167 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 18906
I20260620 20:11:19.344298 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-1/wal/instance
I20260620 20:11:19.344594 5486 external_mini_cluster.cc:1428] Setting key 1c9015983fb9cfdc37da004a5d6141ff
I20260620 20:11:19.346367 18974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
WARNING: no policy specified for kudu/127.5.91.188@KRBTEST.COM; defaulting to no policy
I20260620 20:11:19.351814 18974 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40449 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.188@KRBTEST.COM" created.
Jun 20 20:11:19 dist-test-slave-l8h4 krb5kdc[18817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986279, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.189@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:11:19.366078 18898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } attempt: 3
I20260620 20:11:19.367274 18901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.353785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38824 (local address 127.5.91.190:40449)
0620 20:11:19.353922 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:19.353925 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:19.354767 (+ 842us) server_negotiation.cc:407] Connection header received
0620 20:11:19.355717 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:19.355721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:19.355781 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:19.355882 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:19.357419 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.357966 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.359632 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.359870 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.359957 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.362848 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:19.362851 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:19.362853 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:19.362882 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:19.364529 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.365022 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.365025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.365026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.365104 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.365487 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.365491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.365492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.365745 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:19.365917 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:19.366880 (+ 963us) server_negotiation.cc:299] Negotiation successful
0620 20:11:19.367138 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 20:11:19.368070 18977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.354082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:40449 (local address 127.0.0.1:38824)
0620 20:11:19.354580 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.354661 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:19.355498 (+ 837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:19.356083 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:19.356092 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:19.356585 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:19.357247 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:19.357264 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.358094 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.358098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:19.359479 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:19.359492 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.360065 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.360070 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:19.360273 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.360928 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:19.360949 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:19.362644 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:19.364633 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:19.364639 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:19.364656 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:19.364909 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:19.365197 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:19.365199 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:19.365201 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:19.365352 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:19.365925 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:19.365930 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:19.366333 (+ 403us) client_negotiation.cc:769] Sending connection context
0620 20:11:19.366539 (+ 206us) client_negotiation.cc:240] Negotiation successful
0620 20:11:19.366921 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":129,"threads_started":1}
I20260620 20:11:19.369553 18974 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.188.keytab.
Entry for principal kudu/127.5.91.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.188.keytab.
I20260620 20:11:19.370872 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.188:34493
--webserver_interface=127.5.91.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.188.keytab
--principal=kudu/127.5.91.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:42229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:40449,127.5.91.189:44473,127.5.91.188:34493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 20:11:19 dist-test-slave-l8h4 krb5kdc[18817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986279, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for kudu/127.5.91.189@KRBTEST.COM
Jun 20 20:11:19 dist-test-slave-l8h4 krb5kdc[18817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986279, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.189@KRBTEST.COM for kudu/127.5.91.189@KRBTEST.COM
I20260620 20:11:19.377321 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.366592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:44473 (local address 127.0.0.1:47620)
0620 20:11:19.366788 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.366803 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:19.366880 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:19.367860 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:19.367863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:19.368026 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:19.368652 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:19.368661 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.369987 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.369991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:19.370807 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:19.370818 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.371629 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.371632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:19.371739 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.372227 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:19.372238 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:19.373373 (+ 1135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:19.375553 (+ 2180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:19.375557 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:19.375559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:19.375817 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:19.376153 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:19.376156 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:19.376158 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:19.376249 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:19.376758 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:19.376762 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:19.376872 (+ 110us) client_negotiation.cc:769] Sending connection context
0620 20:11:19.376987 (+ 115us) client_negotiation.cc:240] Negotiation successful
0620 20:11:19.377146 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260620 20:11:19.377671 18979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.367083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47620 (local address 127.5.91.189:44473)
0620 20:11:19.367456 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:19.367466 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:19.367488 (+ 22us) server_negotiation.cc:407] Connection header received
0620 20:11:19.367541 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:19.367547 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:19.367737 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:19.367979 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:19.368764 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.369796 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.370936 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.371501 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.371629 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.373524 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:19.373534 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:19.373550 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:19.373577 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:19.375445 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.375933 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.375940 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.375942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.376057 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.376326 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.376330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.376332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.376640 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:19.376771 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:19.377093 (+ 322us) server_negotiation.cc:299] Negotiation successful
0620 20:11:19.377234 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":131,"threads_started":1}
I20260620 20:11:19.378700 18977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.370045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:44473 (local address 127.0.0.1:47634)
0620 20:11:19.370171 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.370183 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:19.370253 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:19.370703 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:19.370705 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:19.370855 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:19.370946 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:19.370951 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.371850 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.371853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:19.372594 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:19.372605 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.373104 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.373107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:19.373189 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.373609 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:19.373620 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:19.374906 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:19.377196 (+ 2290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:19.377200 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:19.377202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:19.377555 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:19.377814 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:19.377816 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:19.377818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:19.377883 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:19.378237 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:19.378240 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:19.378324 (+ 84us) client_negotiation.cc:769] Sending connection context
0620 20:11:19.378407 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 20:11:19.378545 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260620 20:11:19.378746 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.370250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47634 (local address 127.5.91.189:44473)
0620 20:11:19.370535 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:19.370541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:19.370551 (+ 10us) server_negotiation.cc:407] Connection header received
0620 20:11:19.370575 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:19.370576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:19.370607 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:19.370676 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:19.371057 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.371771 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.372706 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.373009 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.373380 (+ 371us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.375018 (+ 1638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:19.375021 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:19.375023 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:19.375046 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:19.377102 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.377641 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.377643 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.377645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.377729 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.377962 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.377965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.377966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.378130 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:19.378248 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:19.378458 (+ 210us) server_negotiation.cc:299] Negotiation successful
0620 20:11:19.378600 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":228,"spinlock_wait_cycles":1280,"thread_start_us":109,"threads_started":1}
I20260620 20:11:19.379786 18898 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } has no permanent_uuid. Determining permanent_uuid...
I20260620 20:11:19.379853 18974 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } has no permanent_uuid. Determining permanent_uuid...
I20260620 20:11:19.380512 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.380243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:34493 (local address 127.0.0.1:36406)
0620 20:11:19.380379 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.380439 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:34493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260620 20:11:19.380541 18977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.380282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:34493 (local address 127.0.0.1:36418)
0620 20:11:19.380424 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.380484 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:34493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260620 20:11:19.380733 18925 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.91.188:34493: connect: Connection refused (error 111)
I20260620 20:11:19.381412 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.381097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:34493 (local address 127.0.0.1:36432)
0620 20:11:19.381295 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.381344 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:34493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260620 20:11:19.381458 18977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.381268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:34493 (local address 127.0.0.1:36446)
0620 20:11:19.381356 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.381403 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:34493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 20:11:19.381538 18898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.188:34493: Network error: Client connection negotiation failed: client connection to 127.5.91.188:34493: connect: Connection refused (error 111)
W20260620 20:11:19.381623 18974 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.188:34493: Network error: Client connection negotiation failed: client connection to 127.5.91.188:34493: connect: Connection refused (error 111)
I20260620 20:11:19.412711 18898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } attempt: 1
I20260620 20:11:19.414256 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.413630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:34493 (local address 127.0.0.1:36452)
0620 20:11:19.413993 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.414105 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:34493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260620 20:11:19.415297 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.415012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:34493 (local address 127.0.0.1:36460)
0620 20:11:19.415146 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.415214 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:34493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260620 20:11:19.415508 18898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.188:34493: Network error: Client connection negotiation failed: client connection to 127.5.91.188:34493: connect: Connection refused (error 111)
I20260620 20:11:19.430888 18974 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } attempt: 1
I20260620 20:11:19.431874 18977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.431486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:34493 (local address 127.0.0.1:36472)
0620 20:11:19.431702 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.431779 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:34493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260620 20:11:19.432991 18977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.432585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:34493 (local address 127.0.0.1:36482)
0620 20:11:19.432816 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.432896 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:34493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260620 20:11:19.433199 18974 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.188:34493: Network error: Client connection negotiation failed: client connection to 127.5.91.188:34493: connect: Connection refused (error 111)
W20260620 20:11:19.484304 18981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:19.484566 18981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:19.484606 18981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:19.488644 18981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:11:19.488703 18981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:19.488721 18981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:19.488737 18981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:11:19.488752 18981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:11:19.490767 18898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } attempt: 2
I20260620 20:11:19.491891 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.491546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:34493 (local address 127.0.0.1:36484)
0620 20:11:19.491721 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.491789 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:34493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260620 20:11:19.492617 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.492431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:34493 (local address 127.0.0.1:36492)
0620 20:11:19.492520 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.492564 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:34493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 20:11:19.492749 18898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.188:34493: Network error: Client connection negotiation failed: client connection to 127.5.91.188:34493: connect: Connection refused (error 111)
I20260620 20:11:19.494339 18981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:42229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:40449,127.5.91.189:44473,127.5.91.188:34493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.188.keytab
--principal=kudu/127.5.91.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.188:34493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.91.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:19.495333 18981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:19.496318 18981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 20:11:19.501410 18974 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } attempt: 2
I20260620 20:11:19.502130 18977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.501843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:34493 (local address 127.0.0.1:36506)
0620 20:11:19.501967 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.502023 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:34493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260620 20:11:19.502756 18977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.502575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:34493 (local address 127.0.0.1:36516)
0620 20:11:19.502663 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.502706 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:34493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 20:11:19.502861 18974 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.188:34493: Network error: Client connection negotiation failed: client connection to 127.5.91.188:34493: connect: Connection refused (error 111)
W20260620 20:11:19.504503 18988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:19.504534 18990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:19.504501 18987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:19.504909 18981 server_base.cc:1034] running on GCE node
I20260620 20:11:19.505419 18981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:19.506511 18981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:19.507675 18981 hybrid_clock.cc:648] HybridClock initialized: now 1781986279507653 us; error 30 us; skew 500 ppm
Jun 20 20:11:19 dist-test-slave-l8h4 krb5kdc[18817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986279, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:19.510522 18981 init.cc:377] Logged in from keytab as kudu/127.5.91.188@KRBTEST.COM (short username kudu)
I20260620 20:11:19.511413 18981 webserver.cc:460] Webserver started at http://127.5.91.188:33175/ using document root <none> and password file <none>
I20260620 20:11:19.511932 18981 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:19.512001 18981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:19.512216 18981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:11:19.514326 18981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-2/data/instance:
uuid: "6c17f93bd3b341098599f5d70fc4b3f5"
format_stamp: "Formatted at 2026-06-20 20:11:19 on dist-test-slave-l8h4"
server_key: "ada2219792e5e0b9004362ac6d52ded6"
server_key_iv: "f7005ededdf6aaf791bf9ab5c618e51c"
server_key_version: "encryptionkey@0"
I20260620 20:11:19.514866 18981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-2/wal/instance:
uuid: "6c17f93bd3b341098599f5d70fc4b3f5"
format_stamp: "Formatted at 2026-06-20 20:11:19 on dist-test-slave-l8h4"
server_key: "ada2219792e5e0b9004362ac6d52ded6"
server_key_iv: "f7005ededdf6aaf791bf9ab5c618e51c"
server_key_version: "encryptionkey@0"
I20260620 20:11:19.518669 18981 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 20:11:19.521060 18997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:19.522343 18981 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:11:19.522460 18981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-2/wal
uuid: "6c17f93bd3b341098599f5d70fc4b3f5"
format_stamp: "Formatted at 2026-06-20 20:11:19 on dist-test-slave-l8h4"
server_key: "ada2219792e5e0b9004362ac6d52ded6"
server_key_iv: "f7005ededdf6aaf791bf9ab5c618e51c"
server_key_version: "encryptionkey@0"
I20260620 20:11:19.522552 18981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:19.540946 18981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:19.544132 18981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:19.544327 18981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:19.551362 18981 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.188:34493
I20260620 20:11:19.551692 19049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.188:34493 every 8 connection(s)
I20260620 20:11:19.552403 18981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-2/data/info.pb
I20260620 20:11:19.555223 19050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:11:19.558113 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 18981
I20260620 20:11:19.558259 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781986111135528-5486-0/minicluster-data/master-2/wal/instance
I20260620 20:11:19.558518 5486 external_mini_cluster.cc:1428] Setting key 87880bbdb8cfca932a6948864778f4fc
I20260620 20:11:19.559051 19050 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40449 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 20:11:19 dist-test-slave-l8h4 krb5kdc[18817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986279, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.188@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
Jun 20 20:11:19 dist-test-slave-l8h4 krb5kdc[18817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.188@KRBTEST.COM
I20260620 20:11:19.571403 18901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.560797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38840 (local address 127.5.91.190:40449)
0620 20:11:19.560930 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:19.560934 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:19.561302 (+ 368us) server_negotiation.cc:407] Connection header received
0620 20:11:19.561958 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:19.561960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:19.562039 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:19.562108 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:19.563061 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.563553 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.564634 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.564896 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.564970 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.567607 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:19.567610 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:19.567612 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:19.567634 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:19.569132 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.569672 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.569676 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.569678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.569764 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.570215 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.570218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.570219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.570436 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:19.570539 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:19.571111 (+ 572us) server_negotiation.cc:299] Negotiation successful
0620 20:11:19.571238 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260620 20:11:19.572297 19054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.560725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:40449 (local address 127.0.0.1:38840)
0620 20:11:19.561149 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.561197 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:19.561770 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:19.562210 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:19.562216 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:19.562641 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:19.562905 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:19.562917 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.563657 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.563660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:19.564507 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:19.564518 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.565024 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.565028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:19.565235 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.565893 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:19.565914 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:19.567437 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:19.569251 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:19.569259 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:19.569271 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:19.569576 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:19.569863 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:19.569866 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:19.569868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:19.570069 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:19.570541 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:19.570546 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:19.570871 (+ 325us) client_negotiation.cc:769] Sending connection context
0620 20:11:19.571024 (+ 153us) client_negotiation.cc:240] Negotiation successful
0620 20:11:19.571333 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":62,"spinlock_wait_cycles":5888,"thread_start_us":140,"threads_started":1}
I20260620 20:11:19.572309 19053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.560134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36522 (local address 127.5.91.188:34493)
0620 20:11:19.560888 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:19.560910 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:19.560956 (+ 46us) server_negotiation.cc:407] Connection header received
0620 20:11:19.561763 (+ 807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:19.561784 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:19.562008 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:19.562247 (+ 239us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:19.562913 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.563887 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.564727 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.565183 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.565276 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.568033 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:19.568039 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:19.568051 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:19.568077 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:19.569726 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.570265 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.570270 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.570271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.570347 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.570618 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.570621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.570623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.570893 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:19.571126 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:19.571322 (+ 196us) server_negotiation.cc:299] Negotiation successful
0620 20:11:19.571432 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":419,"thread_start_us":197,"threads_started":1}
I20260620 20:11:19.573508 19050 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 20:11:19 dist-test-slave-l8h4 krb5kdc[18817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986279, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.188@KRBTEST.COM for kudu/127.5.91.189@KRBTEST.COM
Jun 20 20:11:19 dist-test-slave-l8h4 krb5kdc[18817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:11:19.581319 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.574118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47650 (local address 127.5.91.189:44473)
0620 20:11:19.574291 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:19.574295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:19.574425 (+ 130us) server_negotiation.cc:407] Connection header received
0620 20:11:19.574529 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:19.574532 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:19.574577 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:19.574634 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:19.574992 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.575511 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.576237 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.576469 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.576556 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.578460 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:19.578463 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:19.578465 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:19.578506 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:19.579782 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.580185 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.580188 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.580189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.580264 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.580525 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.580528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.580530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.580725 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:19.580800 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:19.581101 (+ 301us) server_negotiation.cc:299] Negotiation successful
0620 20:11:19.581203 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260620 20:11:19.581324 19054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.573984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:44473 (local address 127.0.0.1:47650)
0620 20:11:19.574359 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.574370 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:19.574445 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:19.574655 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:19.574658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:19.574802 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:19.574887 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:19.574891 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.575631 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.575634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:19.576137 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:19.576144 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.576567 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.576571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:19.576658 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.577060 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:19.577072 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:19.578287 (+ 1215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:19.579880 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:19.579883 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:19.579885 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:19.580085 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:19.580349 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:19.580353 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:19.580355 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:19.580435 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:19.580816 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:19.580819 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:19.580920 (+ 101us) client_negotiation.cc:769] Sending connection context
0620 20:11:19.581032 (+ 112us) client_negotiation.cc:240] Negotiation successful
0620 20:11:19.581158 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323}
I20260620 20:11:19.582176 19050 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } has no permanent_uuid. Determining permanent_uuid...
I20260620 20:11:19.582739 18901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.574652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38848 (local address 127.5.91.190:40449)
0620 20:11:19.574756 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:19.574760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:19.574823 (+ 63us) server_negotiation.cc:407] Connection header received
0620 20:11:19.574962 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:19.574965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:19.575003 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:19.575068 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:19.575686 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.576165 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.576904 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.577117 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.577204 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.579715 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:19.579719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:19.579721 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:19.579746 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:19.581112 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.581533 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.581537 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.581547 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.581622 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.581858 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.581862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.581864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.582114 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:19.582341 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:19.582484 (+ 143us) server_negotiation.cc:299] Negotiation successful
0620 20:11:19.582600 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Jun 20 20:11:19 dist-test-slave-l8h4 krb5kdc[18817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986279, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.188@KRBTEST.COM for kudu/127.5.91.188@KRBTEST.COM
Jun 20 20:11:19 dist-test-slave-l8h4 krb5kdc[18817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.189@KRBTEST.COM
I20260620 20:11:19.590356 19054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.582663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:34493 (local address 127.0.0.1:36528)
0620 20:11:19.582767 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.582780 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:19.583068 (+ 288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:19.583296 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:19.583299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:19.583444 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:19.583522 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:19.583528 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.584440 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.584443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:19.585064 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:19.585075 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.585597 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.585599 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:19.585676 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.586051 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:19.586063 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:19.587405 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:19.588937 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:19.588939 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:19.588941 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:19.589145 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:19.589387 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:19.589390 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:19.589391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:19.589461 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:19.589813 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:19.589815 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:19.589897 (+ 82us) client_negotiation.cc:769] Sending connection context
0620 20:11:19.590004 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 20:11:19.590123 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":2}
I20260620 20:11:19.590372 19053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.582846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36528 (local address 127.5.91.188:34493)
0620 20:11:19.582932 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:19.582936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:19.582949 (+ 13us) server_negotiation.cc:407] Connection header received
0620 20:11:19.583154 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:19.583157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:19.583193 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:19.583258 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:19.583639 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.584335 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.585184 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.585483 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.585594 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.587536 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:19.587540 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:19.587543 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:19.587567 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:19.588842 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.589233 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.589236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.589237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.589303 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.589563 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.589566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.589568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.589726 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:19.589810 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:19.590121 (+ 311us) server_negotiation.cc:299] Negotiation successful
0620 20:11:19.590251 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":26}
I20260620 20:11:19.591894 18898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } attempt: 3
I20260620 20:11:19.592231 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.583746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47656 (local address 127.5.91.189:44473)
0620 20:11:19.583844 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:19.583847 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:19.584001 (+ 154us) server_negotiation.cc:407] Connection header received
0620 20:11:19.584106 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:19.584108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:19.584142 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:19.584199 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:19.584794 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.585440 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.586303 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.586611 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.586701 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.589119 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:19.589122 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:19.589125 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:19.589147 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:19.590632 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.591086 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.591089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.591090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.591163 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.591410 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.591413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.591415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.591638 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:19.591820 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:19.591976 (+ 156us) server_negotiation.cc:299] Negotiation successful
0620 20:11:19.592090 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 20:11:19.594018 18974 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } attempt: 3
I20260620 20:11:19.594249 19050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5: Bootstrap starting.
I20260620 20:11:19.595260 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
Jun 20 20:11:19 dist-test-slave-l8h4 krb5kdc[18817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986279, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for kudu/127.5.91.188@KRBTEST.COM
I20260620 20:11:19.596993 19050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5: Neither blocks nor log segments found. Creating new log.
I20260620 20:11:19.597936 19050 log.cc:824] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5: Log is configured to *not* fsync() on all Append() calls
I20260620 20:11:19.600576 19050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5: No bootstrap required, opened a new log
I20260620 20:11:19.600679 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.592285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:34493 (local address 127.0.0.1:36544)
0620 20:11:19.592384 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.592398 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:19.592468 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:19.592889 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:19.592892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:19.593065 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:19.593166 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:19.593173 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.593947 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.593951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:19.594699 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:19.594709 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.595195 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.595199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:19.595281 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.595691 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:19.595705 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:19.596906 (+ 1201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:19.599177 (+ 2271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:19.599181 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:19.599183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:19.599417 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:19.599699 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:19.599702 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:19.599704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:19.599787 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:19.600209 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:19.600212 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:19.600305 (+ 93us) client_negotiation.cc:769] Sending connection context
0620 20:11:19.600376 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 20:11:19.600502 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260620 20:11:19.601383 19053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.592593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36544 (local address 127.5.91.188:34493)
0620 20:11:19.592690 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:19.592693 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:19.592704 (+ 11us) server_negotiation.cc:407] Connection header received
0620 20:11:19.592735 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:19.592737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:19.592774 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:19.592844 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:19.593275 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.593814 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.594818 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.595091 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.595178 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.597026 (+ 1848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:19.597029 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:19.597032 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:19.597054 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:19.599065 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.599524 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.599527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.599529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.599606 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.599882 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.599885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.599887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.600109 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:19.601066 (+ 957us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:19.601138 (+ 72us) server_negotiation.cc:299] Negotiation successful
0620 20:11:19.601245 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
Jun 20 20:11:19 dist-test-slave-l8h4 krb5kdc[18817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986279, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.189@KRBTEST.COM for kudu/127.5.91.188@KRBTEST.COM
I20260620 20:11:19.604529 18898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9: Bootstrap starting.
I20260620 20:11:19.605204 19050 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8393b8ea6c40403ea1afb28622f595c9" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40449 } } peers { permanent_uuid: "8b2c551cde6f4d37879989259c70a69c" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } } peers { permanent_uuid: "6c17f93bd3b341098599f5d70fc4b3f5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } }
I20260620 20:11:19.605528 19050 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:11:19.605616 19050 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c17f93bd3b341098599f5d70fc4b3f5, State: Initialized, Role: FOLLOWER
I20260620 20:11:19.606173 19050 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8393b8ea6c40403ea1afb28622f595c9" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40449 } } peers { permanent_uuid: "8b2c551cde6f4d37879989259c70a69c" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } } peers { permanent_uuid: "6c17f93bd3b341098599f5d70fc4b3f5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } }
I20260620 20:11:19.607118 18977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.594353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:34493 (local address 127.0.0.1:36546)
0620 20:11:19.594488 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:19.594501 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:19.594561 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:19.597856 (+ 3295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:19.597860 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:19.598025 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:19.598124 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:19.598131 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.599342 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.599345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:19.600089 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:19.600099 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.600696 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.600699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:19.600778 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.601195 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:19.601208 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:19.602975 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:19.605371 (+ 2396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:19.605374 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:19.605376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:19.605612 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:19.605936 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:19.605940 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:19.605942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:19.606026 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:19.606439 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:19.606442 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:19.606542 (+ 100us) client_negotiation.cc:769] Sending connection context
0620 20:11:19.606620 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 20:11:19.606743 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260620 20:11:19.607288 18898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9: Neither blocks nor log segments found. Creating new log.
I20260620 20:11:19.607677 19050 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:11:19.607952 19055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.594715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36546 (local address 127.5.91.188:34493)
0620 20:11:19.597638 (+ 2923us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:19.597647 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:19.597660 (+ 13us) server_negotiation.cc:407] Connection header received
0620 20:11:19.597690 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:19.597693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:19.597731 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:19.597819 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:19.598475 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.599230 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.600215 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.600574 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.603256 (+ 2682us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.603379 (+ 123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:19.603382 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:19.603384 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:19.603400 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:19.605263 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.605735 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.605738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.605740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.605837 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.606128 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.606131 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.606151 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.606335 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:19.607617 (+ 1282us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:19.607691 (+ 74us) server_negotiation.cc:299] Negotiation successful
0620 20:11:19.607797 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2861,"thread_start_us":101,"threads_started":1}
I20260620 20:11:19.608139 18898 log.cc:824] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9: Log is configured to *not* fsync() on all Append() calls
I20260620 20:11:19.609958 19065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8393b8ea6c40403ea1afb28622f595c9" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40449 } } peers { permanent_uuid: "8b2c551cde6f4d37879989259c70a69c" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } } peers { permanent_uuid: "6c17f93bd3b341098599f5d70fc4b3f5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } } }
I20260620 20:11:19.610184 19065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:11:19.610266 18898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9: No bootstrap required, opened a new log
I20260620 20:11:19.611057 18974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c: Bootstrap starting.
W20260620 20:11:19.612519 19077 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 20:11:19.612586 19077 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 20:11:19.614711 18898 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8393b8ea6c40403ea1afb28622f595c9" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40449 } } peers { permanent_uuid: "8b2c551cde6f4d37879989259c70a69c" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } } peers { permanent_uuid: "6c17f93bd3b341098599f5d70fc4b3f5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } }
I20260620 20:11:19.614884 18974 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c: Neither blocks nor log segments found. Creating new log.
I20260620 20:11:19.615049 18898 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:11:19.615149 18898 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8393b8ea6c40403ea1afb28622f595c9, State: Initialized, Role: FOLLOWER
I20260620 20:11:19.615720 18898 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8393b8ea6c40403ea1afb28622f595c9" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40449 } } peers { permanent_uuid: "8b2c551cde6f4d37879989259c70a69c" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } } peers { permanent_uuid: "6c17f93bd3b341098599f5d70fc4b3f5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } }
I20260620 20:11:19.616219 19079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8393b8ea6c40403ea1afb28622f595c9" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40449 } } peers { permanent_uuid: "8b2c551cde6f4d37879989259c70a69c" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } } peers { permanent_uuid: "6c17f93bd3b341098599f5d70fc4b3f5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } } }
I20260620 20:11:19.616407 19079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:11:19.617007 18898 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9 [sys.catalog]: configured and running, proceeding with master startup.
W20260620 20:11:19.619464 18851 tablet.cc:2369] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:11:19.615537 18981 master_runner.cc:186] Error getting master registration for 127.5.91.190:40449: OK, instance_id { permanent_uuid: "8393b8ea6c40403ea1afb28622f595c9" instance_seqno: 1781986279122954 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 20:11:19.619648 18981 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 20:11:19.620095 18974 log.cc:824] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c: Log is configured to *not* fsync() on all Append() calls
I20260620 20:11:19.622341 19060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.597480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36548 (local address 127.5.91.188:34493)
0620 20:11:19.598594 (+ 1114us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:19.598602 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:19.599077 (+ 475us) server_negotiation.cc:407] Connection header received
0620 20:11:19.599113 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:19.599116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:19.599152 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:19.599224 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:19.600093 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.600814 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.601864 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.602185 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.602272 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.604738 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:19.604742 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:19.604745 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:19.604768 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:19.607583 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.608031 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.608035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.608037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.608114 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.608398 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.608401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.608403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.608602 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:19.608720 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:19.621928 (+ 13208us) server_negotiation.cc:299] Negotiation successful
0620 20:11:19.622098 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1059,"thread_start_us":99,"threads_started":1}
I20260620 20:11:19.622761 18830 master_runner.cc:186] Error getting master registration for 127.5.91.189:44473: OK, instance_id { permanent_uuid: "8b2c551cde6f4d37879989259c70a69c" instance_seqno: 1781986279339462 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 20:11:19.626677 18830 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 20:11:19.628295 18901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.597541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38852 (local address 127.5.91.190:40449)
0620 20:11:19.597648 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:19.597651 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:19.602811 (+ 5160us) server_negotiation.cc:407] Connection header received
0620 20:11:19.603891 (+ 1080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:19.603895 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:19.603938 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:19.603995 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:19.605104 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.605786 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.622414 (+ 16628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.622719 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.622809 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.624623 (+ 1814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:19.624628 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:19.624631 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:19.624656 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:19.626250 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.626812 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.626815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.626817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.626911 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.627187 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.627191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.627192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.627411 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:19.627536 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:19.628047 (+ 511us) server_negotiation.cc:299] Negotiation successful
0620 20:11:19.628156 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 20:11:19.629650 18974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c: No bootstrap required, opened a new log
W20260620 20:11:19.630120 19090 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 20:11:19.630275 19090 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 20:11:19.631925 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:19.598381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47666 (local address 127.5.91.189:44473)
0620 20:11:19.598484 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:19.598489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:19.622462 (+ 23973us) server_negotiation.cc:407] Connection header received
0620 20:11:19.622514 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:19.622517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:19.622570 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:19.622656 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:19.624541 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.625313 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.626484 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:19.626816 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:19.626906 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:19.628443 (+ 1537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:19.628446 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:19.628449 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:19.628475 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:19.629976 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.630456 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.630459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.630461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.630543 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:19.630809 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:19.630812 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:19.630814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:19.631005 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:19.631111 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:19.631668 (+ 557us) server_negotiation.cc:299] Negotiation successful
0620 20:11:19.631783 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 20:11:19.633460 18974 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8393b8ea6c40403ea1afb28622f595c9" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40449 } } peers { permanent_uuid: "8b2c551cde6f4d37879989259c70a69c" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } } peers { permanent_uuid: "6c17f93bd3b341098599f5d70fc4b3f5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } }
I20260620 20:11:19.633622 18974 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:11:19.633667 18974 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c551cde6f4d37879989259c70a69c, State: Initialized, Role: FOLLOWER
I20260620 20:11:19.634086 18974 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8393b8ea6c40403ea1afb28622f595c9" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40449 } } peers { permanent_uuid: "8b2c551cde6f4d37879989259c70a69c" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } } peers { permanent_uuid: "6c17f93bd3b341098599f5d70fc4b3f5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } }
I20260620 20:11:19.634949 19093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8393b8ea6c40403ea1afb28622f595c9" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40449 } } peers { permanent_uuid: "8b2c551cde6f4d37879989259c70a69c" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } } peers { permanent_uuid: "6c17f93bd3b341098599f5d70fc4b3f5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } } }
I20260620 20:11:19.635110 19093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:11:19.635522 18974 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [sys.catalog]: configured and running, proceeding with master startup.
W20260620 20:11:19.641373 19105 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 20:11:19.641510 19105 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 20:11:19.643805 18906 master_runner.cc:263] No leader master found from master 8b2c551cde6f4d37879989259c70a69c
I20260620 20:11:19.643877 18906 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 20:11:19.780550 19093 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:11:19.780783 19093 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8393b8ea6c40403ea1afb28622f595c9" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40449 } } peers { permanent_uuid: "8b2c551cde6f4d37879989259c70a69c" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } } peers { permanent_uuid: "6c17f93bd3b341098599f5d70fc4b3f5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } }
I20260620 20:11:19.781894 19093 leader_election.cc:288] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8393b8ea6c40403ea1afb28622f595c9 (127.5.91.190:40449), 6c17f93bd3b341098599f5d70fc4b3f5 (127.5.91.188:34493)
I20260620 20:11:19.782836 18872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8b2c551cde6f4d37879989259c70a69c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8393b8ea6c40403ea1afb28622f595c9" is_pre_election: true
I20260620 20:11:19.783161 18872 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c551cde6f4d37879989259c70a69c in term 0.
I20260620 20:11:19.783756 18923 leader_election.cc:302] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8393b8ea6c40403ea1afb28622f595c9, 8b2c551cde6f4d37879989259c70a69c; no voters:
I20260620 20:11:19.784152 19093 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:11:19.784242 19093 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:11:19.784283 19093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:11:19.785619 19093 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8393b8ea6c40403ea1afb28622f595c9" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40449 } } peers { permanent_uuid: "8b2c551cde6f4d37879989259c70a69c" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } } peers { permanent_uuid: "6c17f93bd3b341098599f5d70fc4b3f5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } }
I20260620 20:11:19.785986 19093 leader_election.cc:288] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [CANDIDATE]: Term 1 election: Requested vote from peers 8393b8ea6c40403ea1afb28622f595c9 (127.5.91.190:40449), 6c17f93bd3b341098599f5d70fc4b3f5 (127.5.91.188:34493)
I20260620 20:11:19.786442 18872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8b2c551cde6f4d37879989259c70a69c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8393b8ea6c40403ea1afb28622f595c9"
I20260620 20:11:19.786587 18872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:11:19.786979 19024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8b2c551cde6f4d37879989259c70a69c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c17f93bd3b341098599f5d70fc4b3f5" is_pre_election: true
I20260620 20:11:19.787274 19024 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c551cde6f4d37879989259c70a69c in term 0.
I20260620 20:11:19.787992 18872 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c551cde6f4d37879989259c70a69c in term 1.
I20260620 20:11:19.788257 19023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8b2c551cde6f4d37879989259c70a69c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c17f93bd3b341098599f5d70fc4b3f5"
I20260620 20:11:19.788372 19023 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:11:19.790069 19023 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c551cde6f4d37879989259c70a69c in term 1.
I20260620 20:11:19.791790 18923 leader_election.cc:302] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8393b8ea6c40403ea1afb28622f595c9, 8b2c551cde6f4d37879989259c70a69c; no voters:
I20260620 20:11:19.792201 19093 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:11:19.792457 19093 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [term 1 LEADER]: Becoming Leader. State: Replica: 8b2c551cde6f4d37879989259c70a69c, State: Running, Role: LEADER
I20260620 20:11:19.792819 19093 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8393b8ea6c40403ea1afb28622f595c9" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40449 } } peers { permanent_uuid: "8b2c551cde6f4d37879989259c70a69c" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } } peers { permanent_uuid: "6c17f93bd3b341098599f5d70fc4b3f5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } }
I20260620 20:11:19.794675 19093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b2c551cde6f4d37879989259c70a69c. Latest consensus state: current_term: 1 leader_uuid: "8b2c551cde6f4d37879989259c70a69c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8393b8ea6c40403ea1afb28622f595c9" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40449 } } peers { permanent_uuid: "8b2c551cde6f4d37879989259c70a69c" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } } peers { permanent_uuid: "6c17f93bd3b341098599f5d70fc4b3f5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } } }
I20260620 20:11:19.794775 19093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [sys.catalog]: This master's current role is: LEADER
W20260620 20:11:19.795989 19003 tablet.cc:2369] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:11:19.799804 19109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:11:19.800817 19109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:11:19.806583 19023 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5 [term 1 FOLLOWER]: Refusing update from remote peer 8b2c551cde6f4d37879989259c70a69c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:11:19.807655 19093 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c17f93bd3b341098599f5d70fc4b3f5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:11:19.812604 19065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b2c551cde6f4d37879989259c70a69c. Latest consensus state: current_term: 1 leader_uuid: "8b2c551cde6f4d37879989259c70a69c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8393b8ea6c40403ea1afb28622f595c9" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40449 } } peers { permanent_uuid: "8b2c551cde6f4d37879989259c70a69c" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } } peers { permanent_uuid: "6c17f93bd3b341098599f5d70fc4b3f5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } } }
I20260620 20:11:19.812747 19065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:11:19.814186 18872 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9 [term 1 FOLLOWER]: Refusing update from remote peer 8b2c551cde6f4d37879989259c70a69c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:11:19.814862 19107 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8393b8ea6c40403ea1afb28622f595c9" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:11:19.819939 19079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b2c551cde6f4d37879989259c70a69c. Latest consensus state: current_term: 1 leader_uuid: "8b2c551cde6f4d37879989259c70a69c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8393b8ea6c40403ea1afb28622f595c9" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40449 } } peers { permanent_uuid: "8b2c551cde6f4d37879989259c70a69c" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } } peers { permanent_uuid: "6c17f93bd3b341098599f5d70fc4b3f5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } } }
I20260620 20:11:19.820101 19079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:11:19.822654 19107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8b2c551cde6f4d37879989259c70a69c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8393b8ea6c40403ea1afb28622f595c9" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40449 } } peers { permanent_uuid: "8b2c551cde6f4d37879989259c70a69c" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } } peers { permanent_uuid: "6c17f93bd3b341098599f5d70fc4b3f5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } } }
I20260620 20:11:19.824119 19065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b2c551cde6f4d37879989259c70a69c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8393b8ea6c40403ea1afb28622f595c9" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40449 } } peers { permanent_uuid: "8b2c551cde6f4d37879989259c70a69c" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } } peers { permanent_uuid: "6c17f93bd3b341098599f5d70fc4b3f5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } } }
I20260620 20:11:19.824241 19065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:11:19.824826 19107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [sys.catalog]: This master's current role is: LEADER
I20260620 20:11:19.824741 19110 mvcc.cc:205] Tried to move back new op lower bound from 7299015802079571968 to 7299015802034647040. Current Snapshot: MvccSnapshot[applied={T|T < 7299015802079571968}]
I20260620 20:11:19.827351 19109 catalog_manager.cc:1347] Generated new cluster ID: b401a0ecd98a4f50bea09e50e60b0b4b
I20260620 20:11:19.827420 19109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:11:19.829178 19107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8b2c551cde6f4d37879989259c70a69c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8393b8ea6c40403ea1afb28622f595c9" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40449 } } peers { permanent_uuid: "8b2c551cde6f4d37879989259c70a69c" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } } peers { permanent_uuid: "6c17f93bd3b341098599f5d70fc4b3f5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } } }
I20260620 20:11:19.829303 19107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c [sys.catalog]: This master's current role is: LEADER
I20260620 20:11:19.830454 19079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b2c551cde6f4d37879989259c70a69c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8393b8ea6c40403ea1afb28622f595c9" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40449 } } peers { permanent_uuid: "8b2c551cde6f4d37879989259c70a69c" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 44473 } } peers { permanent_uuid: "6c17f93bd3b341098599f5d70fc4b3f5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 34493 } } }
I20260620 20:11:19.830567 19079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:11:19.831435 19113 mvcc.cc:205] Tried to move back new op lower bound from 7299015802079571968 to 7299015802034647040. Current Snapshot: MvccSnapshot[applied={T|T < 7299015802079571968}]
I20260620 20:11:19.847249 19109 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:11:19.848522 19109 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:11:19.865247 19109 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b2c551cde6f4d37879989259c70a69c: Generated new TSK 0
I20260620 20:11:19.866312 19109 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:11:20.614076 19077 catalog_manager.cc:1259] Loaded cluster ID: b401a0ecd98a4f50bea09e50e60b0b4b
I20260620 20:11:20.614207 19077 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5: loading cluster ID for follower catalog manager: success
I20260620 20:11:20.615865 19077 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5: acquiring CA information for follower catalog manager: success
I20260620 20:11:20.617072 19077 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6c17f93bd3b341098599f5d70fc4b3f5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 20:11:20.631584 19090 catalog_manager.cc:1259] Loaded cluster ID: b401a0ecd98a4f50bea09e50e60b0b4b
I20260620 20:11:20.631669 19090 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9: loading cluster ID for follower catalog manager: success
I20260620 20:11:20.633266 19090 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9: acquiring CA information for follower catalog manager: success
I20260620 20:11:20.634425 19090 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8393b8ea6c40403ea1afb28622f595c9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 20:11:20.931447 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 18830
I20260620 20:11:20.936800 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 18906
I20260620 20:11:20.941927 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 18981
2026-06-20T20:11:20Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3096 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:11:21 dist-test-slave-l8h4 krb5kdc[19125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:11:21 dist-test-slave-l8h4 krb5kdc[19125](info): set up 2 sockets
Jun 20 20:11:21 dist-test-slave-l8h4 krb5kdc[19125](info): commencing operation
krb5kdc: starting...
W20260620 20:11:21.935431 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.974s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:11:22 dist-test-slave-l8h4 krb5kdc[19125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:11:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:11:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:11:22.071647 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:44401
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:41437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:44401
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:11:22.219116 19138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:22.220149 19138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:22.220242 19138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:22.226292 19138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:11:22.226395 19138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:22.226418 19138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:22.226435 19138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:11:22.226451 19138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:11:22.233235 19138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:41437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:44401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:44401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:22.235776 19138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:22.237839 19138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:22.247943 19146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:22.248142 19143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:22.248304 19144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:22.248536 19138 server_base.cc:1034] running on GCE node
I20260620 20:11:22.249173 19138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:22.250680 19138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:22.252161 19138 hybrid_clock.cc:648] HybridClock initialized: now 1781986282252114 us; error 71 us; skew 500 ppm
Jun 20 20:11:22 dist-test-slave-l8h4 krb5kdc[19125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986282, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:22.257534 19138 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:11:22.258769 19138 webserver.cc:460] Webserver started at http://127.5.91.190:41017/ using document root <none> and password file <none>
I20260620 20:11:22.260538 19138 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:22.260646 19138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:22.261010 19138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:11:22.263346 19138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "827c59732762473dac6b285ac4adca56"
format_stamp: "Formatted at 2026-06-20 20:11:22 on dist-test-slave-l8h4"
server_key: "d7de3803ba6bae44dcb17dccb8422e78"
server_key_iv: "b5be5d24219563b200d6c1be3b028661"
server_key_version: "encryptionkey@0"
I20260620 20:11:22.263931 19138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "827c59732762473dac6b285ac4adca56"
format_stamp: "Formatted at 2026-06-20 20:11:22 on dist-test-slave-l8h4"
server_key: "d7de3803ba6bae44dcb17dccb8422e78"
server_key_iv: "b5be5d24219563b200d6c1be3b028661"
server_key_version: "encryptionkey@0"
I20260620 20:11:22.267887 19138 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 20:11:22.270951 19153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:22.272496 19138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:11:22.272655 19138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "827c59732762473dac6b285ac4adca56"
format_stamp: "Formatted at 2026-06-20 20:11:22 on dist-test-slave-l8h4"
server_key: "d7de3803ba6bae44dcb17dccb8422e78"
server_key_iv: "b5be5d24219563b200d6c1be3b028661"
server_key_version: "encryptionkey@0"
I20260620 20:11:22.272804 19138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:22.291486 19138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:22.295960 19138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:22.296156 19138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:22.303545 19138 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:44401
I20260620 20:11:22.303606 19205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:44401 every 8 connection(s)
I20260620 20:11:22.304569 19138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:11:22.308791 19206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:11:22.310938 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 19138
I20260620 20:11:22.311103 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:11:22.311401 5486 external_mini_cluster.cc:1428] Setting key fdf412299041846ef69b57e692680452
I20260620 20:11:22.316329 19206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 827c59732762473dac6b285ac4adca56: Bootstrap starting.
I20260620 20:11:22.319407 19206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 827c59732762473dac6b285ac4adca56: Neither blocks nor log segments found. Creating new log.
I20260620 20:11:22.320698 19206 log.cc:824] T 00000000000000000000000000000000 P 827c59732762473dac6b285ac4adca56: Log is configured to *not* fsync() on all Append() calls
Jun 20 20:11:22 dist-test-slave-l8h4 krb5kdc[19125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:11:22.323309 19206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 827c59732762473dac6b285ac4adca56: No bootstrap required, opened a new log
I20260620 20:11:22.328507 19206 raft_consensus.cc:348] T 00000000000000000000000000000000 P 827c59732762473dac6b285ac4adca56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "827c59732762473dac6b285ac4adca56" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44401 } }
I20260620 20:11:22.328837 19206 raft_consensus.cc:374] T 00000000000000000000000000000000 P 827c59732762473dac6b285ac4adca56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:11:22.328902 19206 raft_consensus.cc:729] T 00000000000000000000000000000000 P 827c59732762473dac6b285ac4adca56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 827c59732762473dac6b285ac4adca56, State: Initialized, Role: FOLLOWER
I20260620 20:11:22.329545 19206 consensus_queue.cc:260] T 00000000000000000000000000000000 P 827c59732762473dac6b285ac4adca56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "827c59732762473dac6b285ac4adca56" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44401 } }
I20260620 20:11:22.329727 19206 raft_consensus.cc:388] T 00000000000000000000000000000000 P 827c59732762473dac6b285ac4adca56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:11:22.329779 19206 raft_consensus.cc:482] T 00000000000000000000000000000000 P 827c59732762473dac6b285ac4adca56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:11:22.329876 19206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 827c59732762473dac6b285ac4adca56 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:11:22.331178 19206 raft_consensus.cc:504] T 00000000000000000000000000000000 P 827c59732762473dac6b285ac4adca56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "827c59732762473dac6b285ac4adca56" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44401 } }
I20260620 20:11:22.331575 19209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:22.312869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39696 (local address 127.5.91.190:44401)
0620 20:11:22.313396 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:22.313413 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:22.313454 (+ 41us) server_negotiation.cc:407] Connection header received
0620 20:11:22.314549 (+ 1095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:22.314572 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:22.314951 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:22.315286 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:22.316630 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:22.318161 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:22.319540 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:22.320635 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:22.320835 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:22.323598 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:22.323605 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:22.323626 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:22.323660 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:22.326634 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:22.327890 (+ 1256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:22.327897 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:22.327905 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:22.328080 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:22.328641 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:22.328644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:22.328646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:22.329368 (+ 722us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:22.329609 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:22.330213 (+ 604us) server_negotiation.cc:299] Negotiation successful
0620 20:11:22.330538 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":167,"threads_started":1}
I20260620 20:11:22.331571 19206 leader_election.cc:302] T 00000000000000000000000000000000 P 827c59732762473dac6b285ac4adca56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 827c59732762473dac6b285ac4adca56; no voters:
I20260620 20:11:22.331936 19206 leader_election.cc:288] T 00000000000000000000000000000000 P 827c59732762473dac6b285ac4adca56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:11:22.332108 19211 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 827c59732762473dac6b285ac4adca56 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:11:22.332453 19211 raft_consensus.cc:686] T 00000000000000000000000000000000 P 827c59732762473dac6b285ac4adca56 [term 1 LEADER]: Becoming Leader. State: Replica: 827c59732762473dac6b285ac4adca56, State: Running, Role: LEADER
I20260620 20:11:22.333189 19211 consensus_queue.cc:237] T 00000000000000000000000000000000 P 827c59732762473dac6b285ac4adca56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "827c59732762473dac6b285ac4adca56" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44401 } }
I20260620 20:11:22.333312 19206 sys_catalog.cc:564] T 00000000000000000000000000000000 P 827c59732762473dac6b285ac4adca56 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:11:22.334748 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:11:22.334805 19212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 827c59732762473dac6b285ac4adca56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "827c59732762473dac6b285ac4adca56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "827c59732762473dac6b285ac4adca56" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44401 } } }
I20260620 20:11:22.334918 19212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 827c59732762473dac6b285ac4adca56 [sys.catalog]: This master's current role is: LEADER
I20260620 20:11:22.335335 19213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 827c59732762473dac6b285ac4adca56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 827c59732762473dac6b285ac4adca56. Latest consensus state: current_term: 1 leader_uuid: "827c59732762473dac6b285ac4adca56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "827c59732762473dac6b285ac4adca56" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44401 } } }
I20260620 20:11:22.335448 19213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 827c59732762473dac6b285ac4adca56 [sys.catalog]: This master's current role is: LEADER
I20260620 20:11:22.335927 19222 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:11:22.340478 19222 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:11:22.351271 19222 catalog_manager.cc:1347] Generated new cluster ID: d6e54f9c78f64459be87e37a41429357
I20260620 20:11:22.351428 19222 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:11:22.360339 19222 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:11:22.361528 19222 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:11:22.367659 19222 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 827c59732762473dac6b285ac4adca56: Generated new TSK 0
I20260620 20:11:22.368435 19222 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:11:22.395001 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 19138
2026-06-20T20:11:22Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1453 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:11:22 dist-test-slave-l8h4 krb5kdc[19233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:11:22 dist-test-slave-l8h4 krb5kdc[19233](info): set up 2 sockets
Jun 20 20:11:22 dist-test-slave-l8h4 krb5kdc[19233](info): commencing operation
krb5kdc: starting...
W20260620 20:11:23.391196 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.976s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:11:23 dist-test-slave-l8h4 krb5kdc[19233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:11:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:11:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:11:23.518455 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:46321
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:36163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:46321
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:11:23.637346 19246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:23.637638 19246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:23.637681 19246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:23.641850 19246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:11:23.641937 19246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:23.641955 19246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:23.641971 19246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:11:23.641985 19246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:11:23.647616 19246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:36163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:46321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:46321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:23.648932 19246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:23.650597 19246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 20:11:23.661228 19246 server_base.cc:1034] running on GCE node
W20260620 20:11:23.661170 19251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:23.661259 19252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:23.661376 19254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:23.662184 19246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:23.663421 19246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:23.664601 19246 hybrid_clock.cc:648] HybridClock initialized: now 1781986283664587 us; error 30 us; skew 500 ppm
Jun 20 20:11:23 dist-test-slave-l8h4 krb5kdc[19233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986283, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:23.667421 19246 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:11:23.668426 19246 webserver.cc:460] Webserver started at http://127.5.91.190:44725/ using document root <none> and password file <none>
I20260620 20:11:23.668960 19246 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:23.669019 19246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:23.669190 19246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:11:23.671263 19246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "be988ae9ff8b4530ad335b440f95e011"
format_stamp: "Formatted at 2026-06-20 20:11:23 on dist-test-slave-l8h4"
server_key: "34f43a2f7cd4c6549f5eb1fe6d31501d"
server_key_iv: "d2c26a31c8318b7bf392c621300132e8"
server_key_version: "encryptionkey@0"
I20260620 20:11:23.671759 19246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "be988ae9ff8b4530ad335b440f95e011"
format_stamp: "Formatted at 2026-06-20 20:11:23 on dist-test-slave-l8h4"
server_key: "34f43a2f7cd4c6549f5eb1fe6d31501d"
server_key_iv: "d2c26a31c8318b7bf392c621300132e8"
server_key_version: "encryptionkey@0"
I20260620 20:11:23.675540 19246 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260620 20:11:23.678033 19261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:23.679282 19246 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:11:23.679400 19246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "be988ae9ff8b4530ad335b440f95e011"
format_stamp: "Formatted at 2026-06-20 20:11:23 on dist-test-slave-l8h4"
server_key: "34f43a2f7cd4c6549f5eb1fe6d31501d"
server_key_iv: "d2c26a31c8318b7bf392c621300132e8"
server_key_version: "encryptionkey@0"
I20260620 20:11:23.679494 19246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:23.697530 19246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:23.702591 19246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:23.702780 19246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:23.709988 19246 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:46321
I20260620 20:11:23.710003 19313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:46321 every 8 connection(s)
I20260620 20:11:23.710960 19246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:11:23.713711 19314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:11:23.715253 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 19246
I20260620 20:11:23.715395 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:11:23.715742 5486 external_mini_cluster.cc:1428] Setting key 1ede100556feec7eb5749bd4471b7a37
I20260620 20:11:23.722301 19314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011: Bootstrap starting.
I20260620 20:11:23.725040 19314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011: Neither blocks nor log segments found. Creating new log.
Jun 20 20:11:23 dist-test-slave-l8h4 krb5kdc[19233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:11:23.725833 19314 log.cc:824] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011: Log is configured to *not* fsync() on all Append() calls
I20260620 20:11:23.728066 19314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011: No bootstrap required, opened a new log
I20260620 20:11:23.731743 19314 raft_consensus.cc:348] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be988ae9ff8b4530ad335b440f95e011" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46321 } }
I20260620 20:11:23.731973 19314 raft_consensus.cc:374] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:11:23.732017 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:23.717378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33140 (local address 127.5.91.190:46321)
0620 20:11:23.717858 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:23.717873 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:23.717922 (+ 49us) server_negotiation.cc:407] Connection header received
0620 20:11:23.718757 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:23.718776 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:23.719126 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:23.719437 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:23.720664 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:23.721690 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:23.722896 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:23.723387 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:23.723526 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:23.726263 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:23.726270 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:23.726288 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:23.726322 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:23.728370 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:23.728824 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:23.728829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:23.728833 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:23.728957 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:23.729214 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:23.729217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:23.729218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:23.729965 (+ 747us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:23.730293 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:23.730734 (+ 441us) server_negotiation.cc:299] Negotiation successful
0620 20:11:23.731214 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":139,"threads_started":1}
I20260620 20:11:23.732030 19314 raft_consensus.cc:729] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be988ae9ff8b4530ad335b440f95e011, State: Initialized, Role: FOLLOWER
I20260620 20:11:23.732642 19314 consensus_queue.cc:260] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be988ae9ff8b4530ad335b440f95e011" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46321 } }
I20260620 20:11:23.732781 19314 raft_consensus.cc:388] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:11:23.732867 19314 raft_consensus.cc:482] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:11:23.732963 19314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:11:23.734179 19314 raft_consensus.cc:504] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be988ae9ff8b4530ad335b440f95e011" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46321 } }
I20260620 20:11:23.734530 19314 leader_election.cc:302] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be988ae9ff8b4530ad335b440f95e011; no voters:
I20260620 20:11:23.734956 19314 leader_election.cc:288] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:11:23.734961 19319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:11:23.735229 19319 raft_consensus.cc:686] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 1 LEADER]: Becoming Leader. State: Replica: be988ae9ff8b4530ad335b440f95e011, State: Running, Role: LEADER
I20260620 20:11:23.735558 19319 consensus_queue.cc:237] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be988ae9ff8b4530ad335b440f95e011" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46321 } }
I20260620 20:11:23.736640 19314 sys_catalog.cc:564] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:11:23.737370 19321 sys_catalog.cc:455] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be988ae9ff8b4530ad335b440f95e011" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be988ae9ff8b4530ad335b440f95e011" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46321 } } }
I20260620 20:11:23.737349 19320 sys_catalog.cc:455] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be988ae9ff8b4530ad335b440f95e011. Latest consensus state: current_term: 1 leader_uuid: "be988ae9ff8b4530ad335b440f95e011" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be988ae9ff8b4530ad335b440f95e011" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46321 } } }
I20260620 20:11:23.737577 19321 sys_catalog.cc:458] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [sys.catalog]: This master's current role is: LEADER
I20260620 20:11:23.737588 19320 sys_catalog.cc:458] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [sys.catalog]: This master's current role is: LEADER
I20260620 20:11:23.738622 19324 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:11:23.741870 19324 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:11:23.747706 19324 catalog_manager.cc:1347] Generated new cluster ID: 41d963b2cb5d467d88097c2648c3f038
I20260620 20:11:23.747790 19324 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:11:23.796883 19324 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:11:23.797829 19324 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:11:23.811522 19324 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011: Generated new TSK 0
I20260620 20:11:23.812263 19324 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.91.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
I20260620 20:11:23.841835 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46321
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:36163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:11:23.961577 19340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:23.961860 19340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:23.961907 19340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:23.966497 19340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:23.966558 19340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:23.966619 19340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:11:23.972277 19340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:36163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:23.973323 19340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:23.974398 19340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:23.984139 19345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:23.984210 19348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:23.984169 19346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:23.984611 19340 server_base.cc:1034] running on GCE node
I20260620 20:11:23.985000 19340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:23.985621 19340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:23.986867 19340 hybrid_clock.cc:648] HybridClock initialized: now 1781986283986786 us; error 134 us; skew 500 ppm
Jun 20 20:11:23 dist-test-slave-l8h4 krb5kdc[19233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986283, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:23.989920 19340 init.cc:377] Logged in from keytab as kudu/127.5.91.129@KRBTEST.COM (short username kudu)
I20260620 20:11:23.990818 19340 webserver.cc:460] Webserver started at http://127.5.91.129:39767/ using document root <none> and password file <none>
I20260620 20:11:23.991343 19340 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:23.991403 19340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:23.991590 19340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:11:23.993794 19340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "6c71ee8496d948cb8ed6c8be20f364e7"
format_stamp: "Formatted at 2026-06-20 20:11:23 on dist-test-slave-l8h4"
server_key: "51615fdeb0d0ec0f42a5821b8c88f576"
server_key_iv: "6e075aae5f800989a072c08fa7bb4af3"
server_key_version: "encryptionkey@0"
I20260620 20:11:23.994390 19340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "6c71ee8496d948cb8ed6c8be20f364e7"
format_stamp: "Formatted at 2026-06-20 20:11:23 on dist-test-slave-l8h4"
server_key: "51615fdeb0d0ec0f42a5821b8c88f576"
server_key_iv: "6e075aae5f800989a072c08fa7bb4af3"
server_key_version: "encryptionkey@0"
I20260620 20:11:23.998310 19340 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260620 20:11:24.000813 19355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:24.002099 19340 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 20:11:24.002260 19340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "6c71ee8496d948cb8ed6c8be20f364e7"
format_stamp: "Formatted at 2026-06-20 20:11:23 on dist-test-slave-l8h4"
server_key: "51615fdeb0d0ec0f42a5821b8c88f576"
server_key_iv: "6e075aae5f800989a072c08fa7bb4af3"
server_key_version: "encryptionkey@0"
I20260620 20:11:24.002385 19340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:24.023994 19340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:24.027122 19340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:24.027284 19340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:24.027879 19340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:11:24.028694 19340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:11:24.028740 19340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:24.028784 19340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:11:24.028797 19340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:24.037555 19340 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:42419
I20260620 20:11:24.037640 19468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:42419 every 8 connection(s)
I20260620 20:11:24.038594 19340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:11:24.039393 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 19340
I20260620 20:11:24.039515 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:11:24.039850 5486 external_mini_cluster.cc:1428] Setting key 7b4b75f49afac625688fa831a6a2df5c
WARNING: no policy specified for kudu/127.5.91.130@KRBTEST.COM; defaulting to no policy
Jun 20 20:11:24 dist-test-slave-l8h4 krb5kdc[19233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986283, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.130@KRBTEST.COM" created.
I20260620 20:11:24.054708 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:24.040622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:53827 (local address 127.5.91.190:46321)
0620 20:11:24.040744 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:24.040748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:24.041723 (+ 975us) server_negotiation.cc:407] Connection header received
0620 20:11:24.043012 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:24.043016 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:24.043071 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:24.043160 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:24.044352 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.045153 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.046425 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.046809 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.046930 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:24.050432 (+ 3502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:24.050435 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:24.050437 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:24.050465 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:24.052122 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:24.052852 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:24.052855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:24.052857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:24.052985 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:24.053421 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:24.053424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:24.053425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:24.053634 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:24.053735 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:24.054363 (+ 628us) server_negotiation.cc:299] Negotiation successful
0620 20:11:24.054527 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 20:11:24.055446 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:24.041065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.129:53827)
0620 20:11:24.041557 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:24.041609 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:24.042817 (+ 1208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:24.043363 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:24.043370 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:24.043716 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:24.044203 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:24.044217 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.045285 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.045288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:24.046299 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:24.046311 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.047003 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.047006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:24.047152 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:24.047882 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:24.047907 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:24.050133 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:24.052295 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:24.052304 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:24.052325 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:24.052706 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:24.053079 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:24.053082 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:24.053084 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:24.053273 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:24.053747 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:24.053755 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:24.054071 (+ 316us) client_negotiation.cc:769] Sending connection context
0620 20:11:24.054271 (+ 200us) client_negotiation.cc:240] Negotiation successful
0620 20:11:24.054563 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":127,"threads_started":1}
I20260620 20:11:24.056661 19469 heartbeater.cc:347] Connected to a master server at 127.5.91.190:46321
I20260620 20:11:24.056941 19469 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:24.057579 19469 heartbeater.cc:511] Master 127.5.91.190:46321 requested a full tablet report, sending...
I20260620 20:11:24.060284 19278 ts_manager.cc:194] Registered new tserver with Master: 6c71ee8496d948cb8ed6c8be20f364e7 (127.5.91.129:42419)
I20260620 20:11:24.062244 19278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:53827
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
I20260620 20:11:24.070861 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46321
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:36163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:11:24.187708 19474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:24.188102 19474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:24.188149 19474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:24.193392 19474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:24.193472 19474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:24.193555 19474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:11:24.199388 19474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:36163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:24.200445 19474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:24.201433 19474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 20:11:24.212785 19474 server_base.cc:1034] running on GCE node
W20260620 20:11:24.212510 19480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:24.212510 19479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:24.212517 19482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:24.213717 19474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:24.214751 19474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:24.215978 19474 hybrid_clock.cc:648] HybridClock initialized: now 1781986284215954 us; error 34 us; skew 500 ppm
Jun 20 20:11:24 dist-test-slave-l8h4 krb5kdc[19233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986284, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:24.218931 19474 init.cc:377] Logged in from keytab as kudu/127.5.91.130@KRBTEST.COM (short username kudu)
I20260620 20:11:24.219964 19474 webserver.cc:460] Webserver started at http://127.5.91.130:46335/ using document root <none> and password file <none>
I20260620 20:11:24.220525 19474 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:24.220613 19474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:24.220803 19474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:11:24.222891 19474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "a5e72ce2e215407bbffef86f037239ec"
format_stamp: "Formatted at 2026-06-20 20:11:24 on dist-test-slave-l8h4"
server_key: "58283e8d4c8ed6e0cc0140af805bdd2b"
server_key_iv: "4fa6432d01e57dbe243ecc191b6e4c1c"
server_key_version: "encryptionkey@0"
I20260620 20:11:24.223428 19474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "a5e72ce2e215407bbffef86f037239ec"
format_stamp: "Formatted at 2026-06-20 20:11:24 on dist-test-slave-l8h4"
server_key: "58283e8d4c8ed6e0cc0140af805bdd2b"
server_key_iv: "4fa6432d01e57dbe243ecc191b6e4c1c"
server_key_version: "encryptionkey@0"
I20260620 20:11:24.227471 19474 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260620 20:11:24.230027 19489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:24.231383 19474 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 20:11:24.231490 19474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "a5e72ce2e215407bbffef86f037239ec"
format_stamp: "Formatted at 2026-06-20 20:11:24 on dist-test-slave-l8h4"
server_key: "58283e8d4c8ed6e0cc0140af805bdd2b"
server_key_iv: "4fa6432d01e57dbe243ecc191b6e4c1c"
server_key_version: "encryptionkey@0"
I20260620 20:11:24.231621 19474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:24.243111 19474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:24.246876 19474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:24.247067 19474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:24.248063 19474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:11:24.248932 19474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:11:24.249099 19474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:24.249173 19474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:11:24.249199 19474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:24.260104 19474 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:45065
I20260620 20:11:24.260285 19602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:45065 every 8 connection(s)
I20260620 20:11:24.261140 19474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:11:24.267853 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 19474
I20260620 20:11:24.267957 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:11:24.268211 5486 external_mini_cluster.cc:1428] Setting key 720214a766a4fccae62b6a85aa71f701
Jun 20 20:11:24 dist-test-slave-l8h4 krb5kdc[19233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986284, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.91.131@KRBTEST.COM; defaulting to no policy
I20260620 20:11:24.279151 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:24.263183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:45789 (local address 127.5.91.190:46321)
0620 20:11:24.263390 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:24.263395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:24.264194 (+ 799us) server_negotiation.cc:407] Connection header received
0620 20:11:24.265623 (+ 1429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:24.265626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:24.265685 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:24.265768 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:24.267875 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.268865 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.271107 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.271371 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.271465 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:24.274262 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:24.274267 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:24.274269 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:24.274298 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:24.275898 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:24.276830 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:24.276834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:24.276836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:24.276948 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:24.277431 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:24.277434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:24.277436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:24.277682 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:24.277787 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:24.278842 (+ 1055us) server_negotiation.cc:299] Negotiation successful
0620 20:11:24.278989 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260620 20:11:24.279989 19605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:24.263380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.130:45789)
0620 20:11:24.263958 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:24.264068 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:24.265361 (+ 1293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:24.266220 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:24.266236 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:24.266786 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:24.267589 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:24.267622 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.269136 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.269141 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:24.270872 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:24.270886 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.271529 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.271534 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:24.271704 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:24.272375 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:24.272395 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:24.274056 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:24.276172 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:24.276181 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:24.276203 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:24.276644 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:24.277107 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:24.277110 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:24.277112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:24.277298 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:24.277828 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:24.277835 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:24.278518 (+ 683us) client_negotiation.cc:769] Sending connection context
0620 20:11:24.278726 (+ 208us) client_negotiation.cc:240] Negotiation successful
0620 20:11:24.279079 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":123,"threads_started":1}
I20260620 20:11:24.281592 19603 heartbeater.cc:347] Connected to a master server at 127.5.91.190:46321
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.131@KRBTEST.COM" created.
I20260620 20:11:24.281863 19603 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:24.282475 19603 heartbeater.cc:511] Master 127.5.91.190:46321 requested a full tablet report, sending...
I20260620 20:11:24.283766 19278 ts_manager.cc:194] Registered new tserver with Master: a5e72ce2e215407bbffef86f037239ec (127.5.91.130:45065)
I20260620 20:11:24.284390 19278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:45789
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
I20260620 20:11:24.302995 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46321
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:36163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:11:24.415272 19608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:24.415542 19608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:24.415587 19608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:24.419759 19608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:24.419818 19608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:24.419878 19608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:11:24.425206 19608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:36163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:24.426286 19608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:24.427234 19608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:24.437824 19616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:24.437839 19614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:24.438175 19608 server_base.cc:1034] running on GCE node
W20260620 20:11:24.437851 19613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:24.438635 19608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:24.439335 19608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:24.440512 19608 hybrid_clock.cc:648] HybridClock initialized: now 1781986284440494 us; error 33 us; skew 500 ppm
Jun 20 20:11:24 dist-test-slave-l8h4 krb5kdc[19233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986284, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:24.443238 19608 init.cc:377] Logged in from keytab as kudu/127.5.91.131@KRBTEST.COM (short username kudu)
I20260620 20:11:24.444074 19608 webserver.cc:460] Webserver started at http://127.5.91.131:44899/ using document root <none> and password file <none>
I20260620 20:11:24.444574 19608 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:24.444628 19608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:24.444784 19608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:11:24.446758 19608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "4328eee8a6e4461881c75924c93fc75f"
format_stamp: "Formatted at 2026-06-20 20:11:24 on dist-test-slave-l8h4"
server_key: "6d42255f22252c21e8984a6dce10b814"
server_key_iv: "e292c43b3ed787405bda6c70fd1e6a6d"
server_key_version: "encryptionkey@0"
I20260620 20:11:24.447269 19608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "4328eee8a6e4461881c75924c93fc75f"
format_stamp: "Formatted at 2026-06-20 20:11:24 on dist-test-slave-l8h4"
server_key: "6d42255f22252c21e8984a6dce10b814"
server_key_iv: "e292c43b3ed787405bda6c70fd1e6a6d"
server_key_version: "encryptionkey@0"
I20260620 20:11:24.450884 19608 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 20:11:24.453256 19623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:24.454700 19608 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:11:24.454806 19608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "4328eee8a6e4461881c75924c93fc75f"
format_stamp: "Formatted at 2026-06-20 20:11:24 on dist-test-slave-l8h4"
server_key: "6d42255f22252c21e8984a6dce10b814"
server_key_iv: "e292c43b3ed787405bda6c70fd1e6a6d"
server_key_version: "encryptionkey@0"
I20260620 20:11:24.454895 19608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:24.477677 19608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:24.480823 19608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:24.480978 19608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:24.481498 19608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:11:24.482318 19608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:11:24.482363 19608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:24.482401 19608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:11:24.482420 19608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:24.490893 19608 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:37033
I20260620 20:11:24.490927 19738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:37033 every 8 connection(s)
I20260620 20:11:24.491732 19608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:11:24.500005 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 19608
I20260620 20:11:24.500144 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:11:24.500437 5486 external_mini_cluster.cc:1428] Setting key 47680f75080f060bc2b26047e43a923e
Jun 20 20:11:24 dist-test-slave-l8h4 krb5kdc[19233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986284, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:11:24.505445 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:24.493580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:48173 (local address 127.5.91.190:46321)
0620 20:11:24.493721 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:24.493724 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:24.494572 (+ 848us) server_negotiation.cc:407] Connection header received
0620 20:11:24.495915 (+ 1343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:24.495919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:24.495972 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:24.496047 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:24.497225 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.497754 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.498910 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.499128 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.499213 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:24.501787 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:24.501792 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:24.501795 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:24.501822 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:24.503438 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:24.503970 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:24.503973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:24.503975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:24.504053 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:24.504365 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:24.504368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:24.504369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:24.504568 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:24.504656 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:24.505171 (+ 515us) server_negotiation.cc:299] Negotiation successful
0620 20:11:24.505271 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 20:11:24.506278 19741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:24.493956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.131:48173)
0620 20:11:24.494415 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:24.494456 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:24.495697 (+ 1241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:24.496228 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:24.496234 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:24.496578 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:24.497080 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:24.497094 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.497871 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.497874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:24.498792 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:24.498803 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.499265 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.499269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:24.499426 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:24.500016 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:24.500032 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:24.501631 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:24.503579 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:24.503584 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:24.503597 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:24.503868 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:24.504129 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:24.504132 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:24.504133 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:24.504260 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:24.504690 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:24.504696 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:24.504946 (+ 250us) client_negotiation.cc:769] Sending connection context
0620 20:11:24.505109 (+ 163us) client_negotiation.cc:240] Negotiation successful
0620 20:11:24.505434 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":150,"threads_started":1}
I20260620 20:11:24.507390 19739 heartbeater.cc:347] Connected to a master server at 127.5.91.190:46321
I20260620 20:11:24.507649 19739 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:24.508121 19739 heartbeater.cc:511] Master 127.5.91.190:46321 requested a full tablet report, sending...
I20260620 20:11:24.509263 19278 ts_manager.cc:194] Registered new tserver with Master: 4328eee8a6e4461881c75924c93fc75f (127.5.91.131:37033)
I20260620 20:11:24.509809 19278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:48173
I20260620 20:11:24.514392 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 20:11:24.523592 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:24.516174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33144 (local address 127.5.91.190:46321)
0620 20:11:24.516319 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:24.516323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:24.516443 (+ 120us) server_negotiation.cc:407] Connection header received
0620 20:11:24.516506 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:24.516508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:24.516642 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:24.516721 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:24.517417 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.517945 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.518831 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.519038 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.519124 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:24.520476 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:24.520479 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:24.520481 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:24.520502 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:24.521983 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:24.522422 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:24.522425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:24.522426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:24.522517 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:24.522779 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:24.522782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:24.522783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:24.523046 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:24.523160 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:24.523319 (+ 159us) server_negotiation.cc:299] Negotiation successful
0620 20:11:24.523422 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260620 20:11:24.526512 19278 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:11:24.529403 19278 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:11:24.543814 19750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:24.538075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:45065 (local address 127.0.0.1:54902)
0620 20:11:24.538652 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:24.538701 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:24.538832 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:24.539835 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:24.539841 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:24.539863 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:11:24.540008 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:24.540016 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.541134 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.541138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:24.542231 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:24.542240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.543249 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.543252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:24.543358 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:24.543394 (+ 36us) client_negotiation.cc:769] Sending connection context
0620 20:11:24.543536 (+ 142us) client_negotiation.cc:240] Negotiation successful
0620 20:11:24.543664 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":105,"threads_started":1}
I20260620 20:11:24.543817 19752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:24.538212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54902 (local address 127.5.91.130:45065)
0620 20:11:24.539348 (+ 1136us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:24.539359 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:24.539381 (+ 22us) server_negotiation.cc:407] Connection header received
0620 20:11:24.539457 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:24.539463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:24.539681 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:24.539838 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:11:24.540128 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.541007 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.542348 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.543138 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.543230 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:24.543378 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:24.543573 (+ 195us) server_negotiation.cc:299] Negotiation successful
0620 20:11:24.543665 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1043,"thread_start_us":96,"threads_started":1}
I20260620 20:11:24.544550 19753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:24.538718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57834 (local address 127.5.91.129:42419)
0620 20:11:24.539068 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:24.539084 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:24.539362 (+ 278us) server_negotiation.cc:407] Connection header received
0620 20:11:24.539764 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:24.539770 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:24.539939 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:24.540065 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:11:24.540444 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.541584 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.542967 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.543762 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.543864 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:24.543988 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:24.544294 (+ 306us) server_negotiation.cc:299] Negotiation successful
0620 20:11:24.544385 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":92,"threads_started":1}
I20260620 20:11:24.544739 19755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:24.539047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40494 (local address 127.5.91.131:37033)
0620 20:11:24.539435 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:24.539445 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:24.539621 (+ 176us) server_negotiation.cc:407] Connection header received
0620 20:11:24.539805 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:24.539810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:24.539971 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:24.540108 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:11:24.540338 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.541529 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.543433 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.544208 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.544299 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:24.544416 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:24.544520 (+ 104us) server_negotiation.cc:299] Negotiation successful
0620 20:11:24.544610 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":133,"threads_started":1}
I20260620 20:11:24.544811 19754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:24.538805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:37033 (local address 127.0.0.1:40494)
0620 20:11:24.539451 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:24.539462 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:24.539544 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:24.540117 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:24.540119 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:24.540136 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:11:24.540233 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:24.540240 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.541666 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.541669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:24.543299 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:24.543309 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.544314 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.544318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:24.544400 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:24.544422 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 20:11:24.544482 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 20:11:24.544552 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":660,"thread_start_us":89,"threads_started":1}
I20260620 20:11:24.544920 19751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:24.538183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:42419 (local address 127.0.0.1:57834)
0620 20:11:24.539276 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:24.539289 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:24.539629 (+ 340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:24.540239 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:24.540242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:24.540259 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:11:24.540332 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:24.540338 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.541713 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.541716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:24.542844 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:24.542855 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.544049 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.544053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:24.544141 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:24.544164 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 20:11:24.544686 (+ 522us) client_negotiation.cc:240] Negotiation successful
0620 20:11:24.544762 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1024,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
I20260620 20:11:24.546854 19537 tablet_service.cc:1467] Processing CreateTablet for tablet f5764294c6b34981862995dd720ca6a5 (DEFAULT_TABLE table=test-table [id=30e131160ce74b2da74bf0eb7df138ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:11:24.547240 19403 tablet_service.cc:1467] Processing CreateTablet for tablet f5764294c6b34981862995dd720ca6a5 (DEFAULT_TABLE table=test-table [id=30e131160ce74b2da74bf0eb7df138ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:11:24.547820 19537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5764294c6b34981862995dd720ca6a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:11:24.548077 19403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5764294c6b34981862995dd720ca6a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:11:24.553771 19757 tablet_bootstrap.cc:492] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7: Bootstrap starting.
I20260620 20:11:24.554564 19756 tablet_bootstrap.cc:492] T f5764294c6b34981862995dd720ca6a5 P a5e72ce2e215407bbffef86f037239ec: Bootstrap starting.
I20260620 20:11:24.556278 19757 tablet_bootstrap.cc:654] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7: Neither blocks nor log segments found. Creating new log.
I20260620 20:11:24.556874 19756 tablet_bootstrap.cc:654] T f5764294c6b34981862995dd720ca6a5 P a5e72ce2e215407bbffef86f037239ec: Neither blocks nor log segments found. Creating new log.
I20260620 20:11:24.557102 19757 log.cc:824] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7: Log is configured to *not* fsync() on all Append() calls
I20260620 20:11:24.557571 19756 log.cc:824] T f5764294c6b34981862995dd720ca6a5 P a5e72ce2e215407bbffef86f037239ec: Log is configured to *not* fsync() on all Append() calls
I20260620 20:11:24.557818 19671 tablet_service.cc:1467] Processing CreateTablet for tablet f5764294c6b34981862995dd720ca6a5 (DEFAULT_TABLE table=test-table [id=30e131160ce74b2da74bf0eb7df138ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:11:24.558966 19671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5764294c6b34981862995dd720ca6a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:11:24.559253 19757 tablet_bootstrap.cc:492] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7: No bootstrap required, opened a new log
I20260620 20:11:24.559262 19756 tablet_bootstrap.cc:492] T f5764294c6b34981862995dd720ca6a5 P a5e72ce2e215407bbffef86f037239ec: No bootstrap required, opened a new log
I20260620 20:11:24.559441 19757 ts_tablet_manager.cc:1397] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260620 20:11:24.559443 19756 ts_tablet_manager.cc:1397] T f5764294c6b34981862995dd720ca6a5 P a5e72ce2e215407bbffef86f037239ec: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 20:11:24.562994 19756 raft_consensus.cc:348] T f5764294c6b34981862995dd720ca6a5 P a5e72ce2e215407bbffef86f037239ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } }
I20260620 20:11:24.563265 19756 raft_consensus.cc:374] T f5764294c6b34981862995dd720ca6a5 P a5e72ce2e215407bbffef86f037239ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:11:24.563324 19756 raft_consensus.cc:729] T f5764294c6b34981862995dd720ca6a5 P a5e72ce2e215407bbffef86f037239ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5e72ce2e215407bbffef86f037239ec, State: Initialized, Role: FOLLOWER
I20260620 20:11:24.563741 19756 consensus_queue.cc:260] T f5764294c6b34981862995dd720ca6a5 P a5e72ce2e215407bbffef86f037239ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } }
I20260620 20:11:24.563799 19757 raft_consensus.cc:348] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } }
I20260620 20:11:24.564059 19757 raft_consensus.cc:374] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:11:24.564116 19757 raft_consensus.cc:729] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c71ee8496d948cb8ed6c8be20f364e7, State: Initialized, Role: FOLLOWER
I20260620 20:11:24.564515 19603 heartbeater.cc:503] Master 127.5.91.190:46321 was elected leader, sending a full tablet report...
I20260620 20:11:24.564558 19757 consensus_queue.cc:260] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } }
I20260620 20:11:24.565186 19469 heartbeater.cc:503] Master 127.5.91.190:46321 was elected leader, sending a full tablet report...
I20260620 20:11:24.565551 19757 ts_tablet_manager.cc:1428] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260620 20:11:24.566334 19756 ts_tablet_manager.cc:1428] T f5764294c6b34981862995dd720ca6a5 P a5e72ce2e215407bbffef86f037239ec: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260620 20:11:24.566656 19762 tablet_bootstrap.cc:492] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f: Bootstrap starting.
I20260620 20:11:24.569033 19762 tablet_bootstrap.cc:654] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f: Neither blocks nor log segments found. Creating new log.
I20260620 20:11:24.570168 19762 log.cc:824] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f: Log is configured to *not* fsync() on all Append() calls
I20260620 20:11:24.571985 19762 tablet_bootstrap.cc:492] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f: No bootstrap required, opened a new log
I20260620 20:11:24.572181 19762 ts_tablet_manager.cc:1397] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 20:11:24.575151 19762 raft_consensus.cc:348] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } }
I20260620 20:11:24.575382 19762 raft_consensus.cc:374] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:11:24.575415 19762 raft_consensus.cc:729] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4328eee8a6e4461881c75924c93fc75f, State: Initialized, Role: FOLLOWER
I20260620 20:11:24.575851 19762 consensus_queue.cc:260] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } }
I20260620 20:11:24.576773 19762 ts_tablet_manager.cc:1428] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260620 20:11:24.577135 19739 heartbeater.cc:503] Master 127.5.91.190:46321 was elected leader, sending a full tablet report...
W20260620 20:11:24.742969 19740 tablet.cc:2369] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 20:11:24.762344 19604 tablet.cc:2369] T f5764294c6b34981862995dd720ca6a5 P a5e72ce2e215407bbffef86f037239ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 20:11:24.790042 19470 tablet.cc:2369] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:11:24.794879 19764 raft_consensus.cc:482] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:11:24.795132 19764 raft_consensus.cc:504] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } }
I20260620 20:11:24.796321 19764 leader_election.cc:288] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5e72ce2e215407bbffef86f037239ec (127.5.91.130:45065), 6c71ee8496d948cb8ed6c8be20f364e7 (127.5.91.129:42419)
I20260620 20:11:24.799876 19753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:24.796734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:59559 (local address 127.5.91.129:42419)
0620 20:11:24.796968 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:24.796973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:24.796986 (+ 13us) server_negotiation.cc:407] Connection header received
0620 20:11:24.797037 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:24.797040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:24.797083 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:24.797146 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:11:24.797418 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.797926 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.798981 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.799371 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.799476 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:24.799539 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:24.799702 (+ 163us) server_negotiation.cc:299] Negotiation successful
0620 20:11:24.799767 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260620 20:11:24.799876 19741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:24.796625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:42419 (local address 127.5.91.131:59559)
0620 20:11:24.796760 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:24.796775 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:24.796880 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:24.797187 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:24.797190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:24.797209 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:11:24.797315 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:24.797319 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.798045 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.798049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:24.798859 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:24.798867 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.799482 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.799485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:24.799588 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:24.799608 (+ 20us) client_negotiation.cc:769] Sending connection context
0620 20:11:24.799671 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 20:11:24.799742 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260620 20:11:24.800328 19752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:24.796739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:52613 (local address 127.5.91.130:45065)
0620 20:11:24.796878 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:24.796882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:24.797250 (+ 368us) server_negotiation.cc:407] Connection header received
0620 20:11:24.797378 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:24.797382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:24.797428 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:24.797503 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:11:24.797757 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.798396 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.799406 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.799782 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.799883 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:24.799931 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:24.800142 (+ 211us) server_negotiation.cc:299] Negotiation successful
0620 20:11:24.800214 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260620 20:11:24.800577 19423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5764294c6b34981862995dd720ca6a5" candidate_uuid: "4328eee8a6e4461881c75924c93fc75f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" is_pre_election: true
I20260620 20:11:24.800659 19765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:24.796625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:45065 (local address 127.5.91.131:52613)
0620 20:11:24.797145 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:24.797158 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:24.797273 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:24.797518 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:24.797521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:24.797537 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:11:24.797653 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:24.797659 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.798522 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.798526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:24.799287 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:24.799296 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.799906 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.799909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:24.800002 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:24.800023 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 20:11:24.800456 (+ 433us) client_negotiation.cc:240] Negotiation successful
0620 20:11:24.800530 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":169,"thread_start_us":102,"threads_started":1}
I20260620 20:11:24.800872 19423 raft_consensus.cc:2454] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4328eee8a6e4461881c75924c93fc75f in term 0.
I20260620 20:11:24.801182 19557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5764294c6b34981862995dd720ca6a5" candidate_uuid: "4328eee8a6e4461881c75924c93fc75f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e72ce2e215407bbffef86f037239ec" is_pre_election: true
I20260620 20:11:24.801350 19627 leader_election.cc:302] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4328eee8a6e4461881c75924c93fc75f, 6c71ee8496d948cb8ed6c8be20f364e7; no voters:
I20260620 20:11:24.801522 19557 raft_consensus.cc:2454] T f5764294c6b34981862995dd720ca6a5 P a5e72ce2e215407bbffef86f037239ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4328eee8a6e4461881c75924c93fc75f in term 0.
I20260620 20:11:24.801610 19764 raft_consensus.cc:2781] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:11:24.801692 19764 raft_consensus.cc:482] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:11:24.801739 19764 raft_consensus.cc:3037] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:11:24.802816 19764 raft_consensus.cc:504] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } }
I20260620 20:11:24.803124 19764 leader_election.cc:288] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f [CANDIDATE]: Term 1 election: Requested vote from peers a5e72ce2e215407bbffef86f037239ec (127.5.91.130:45065), 6c71ee8496d948cb8ed6c8be20f364e7 (127.5.91.129:42419)
I20260620 20:11:24.803525 19557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5764294c6b34981862995dd720ca6a5" candidate_uuid: "4328eee8a6e4461881c75924c93fc75f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e72ce2e215407bbffef86f037239ec"
I20260620 20:11:24.803562 19423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5764294c6b34981862995dd720ca6a5" candidate_uuid: "4328eee8a6e4461881c75924c93fc75f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c71ee8496d948cb8ed6c8be20f364e7"
I20260620 20:11:24.803663 19423 raft_consensus.cc:3037] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:11:24.803658 19557 raft_consensus.cc:3037] T f5764294c6b34981862995dd720ca6a5 P a5e72ce2e215407bbffef86f037239ec [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:11:24.804870 19423 raft_consensus.cc:2454] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4328eee8a6e4461881c75924c93fc75f in term 1.
I20260620 20:11:24.804870 19557 raft_consensus.cc:2454] T f5764294c6b34981862995dd720ca6a5 P a5e72ce2e215407bbffef86f037239ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4328eee8a6e4461881c75924c93fc75f in term 1.
I20260620 20:11:24.805130 19627 leader_election.cc:302] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4328eee8a6e4461881c75924c93fc75f, 6c71ee8496d948cb8ed6c8be20f364e7; no voters:
I20260620 20:11:24.805290 19764 raft_consensus.cc:2781] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:11:24.805480 19764 raft_consensus.cc:686] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f [term 1 LEADER]: Becoming Leader. State: Replica: 4328eee8a6e4461881c75924c93fc75f, State: Running, Role: LEADER
I20260620 20:11:24.805832 19764 consensus_queue.cc:237] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } }
I20260620 20:11:24.809725 19277 catalog_manager.cc:5495] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4328eee8a6e4461881c75924c93fc75f (127.5.91.131). New cstate: current_term: 1 leader_uuid: "4328eee8a6e4461881c75924c93fc75f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } health_report { overall_health: HEALTHY } } }
I20260620 20:11:24.841451 19755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:24.838015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40504 (local address 127.5.91.131:37033)
0620 20:11:24.838160 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:24.838164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:24.838181 (+ 17us) server_negotiation.cc:407] Connection header received
0620 20:11:24.838313 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:24.838316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:24.838372 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:24.838462 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:11:24.838759 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.839302 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.840129 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:24.840336 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:24.840411 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:24.840711 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:11:24.841127 (+ 416us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:11:24.841210 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:24.841293 (+ 83us) server_negotiation.cc:299] Negotiation successful
0620 20:11:24.841344 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 20:11:24.847261 19423 raft_consensus.cc:1261] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Refusing update from remote peer 4328eee8a6e4461881c75924c93fc75f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:11:24.847261 19557 raft_consensus.cc:1261] T f5764294c6b34981862995dd720ca6a5 P a5e72ce2e215407bbffef86f037239ec [term 1 FOLLOWER]: Refusing update from remote peer 4328eee8a6e4461881c75924c93fc75f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:11:24.847955 19766 consensus_queue.cc:1035] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:11:24.848130 19764 consensus_queue.cc:1035] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:11:24.855317 19769 mvcc.cc:205] Tried to move back new op lower bound from 7299015822726959104 to 7299015822567460864. Current Snapshot: MvccSnapshot[applied={T|T < 7299015822726959104}]
I20260620 20:11:24.866655 19276 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:11:24.866832 19276 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:11:24.869398 19276 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011: Sending DeleteTablet for 3 replicas of tablet f5764294c6b34981862995dd720ca6a5
I20260620 20:11:24.870254 19537 tablet_service.cc:1514] Processing DeleteTablet for tablet f5764294c6b34981862995dd720ca6a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:11:24 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:54902
I20260620 20:11:24.870252 19671 tablet_service.cc:1514] Processing DeleteTablet for tablet f5764294c6b34981862995dd720ca6a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:11:24 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:40494
I20260620 20:11:24.870265 19403 tablet_service.cc:1514] Processing DeleteTablet for tablet f5764294c6b34981862995dd720ca6a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:11:24 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:57834
I20260620 20:11:24.870790 19779 tablet_replica.cc:330] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f: stopping tablet replica
I20260620 20:11:24.870904 19780 tablet_replica.cc:330] T f5764294c6b34981862995dd720ca6a5 P a5e72ce2e215407bbffef86f037239ec: stopping tablet replica
I20260620 20:11:24.870931 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 19340
I20260620 20:11:24.872471 19779 raft_consensus.cc:2229] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f [term 1 LEADER]: Raft consensus shutting down.
I20260620 20:11:24.872887 19779 raft_consensus.cc:2258] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:11:24.874361 19780 raft_consensus.cc:2229] T f5764294c6b34981862995dd720ca6a5 P a5e72ce2e215407bbffef86f037239ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:11:24.874738 19780 raft_consensus.cc:2258] T f5764294c6b34981862995dd720ca6a5 P a5e72ce2e215407bbffef86f037239ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:11:24.874884 19779 ts_tablet_manager.cc:1905] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:11:24.876338 19780 ts_tablet_manager.cc:1905] T f5764294c6b34981862995dd720ca6a5 P a5e72ce2e215407bbffef86f037239ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:11:24.877624 19779 ts_tablet_manager.cc:1918] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:11:24.877718 19779 log.cc:1196] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/f5764294c6b34981862995dd720ca6a5
I20260620 20:11:24.878063 19779 ts_tablet_manager.cc:1939] T f5764294c6b34981862995dd720ca6a5 P 4328eee8a6e4461881c75924c93fc75f: Deleting consensus metadata
W20260620 20:11:24.879040 19265 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.91.129:42419 (error 108)
I20260620 20:11:24.879099 19780 ts_tablet_manager.cc:1918] T f5764294c6b34981862995dd720ca6a5 P a5e72ce2e215407bbffef86f037239ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:11:24.879304 19780 log.cc:1196] T f5764294c6b34981862995dd720ca6a5 P a5e72ce2e215407bbffef86f037239ec: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/wal/wals/f5764294c6b34981862995dd720ca6a5
I20260620 20:11:24.879266 19264 catalog_manager.cc:4841] TS 4328eee8a6e4461881c75924c93fc75f (127.5.91.131:37033): tablet f5764294c6b34981862995dd720ca6a5 (table test-table [id=30e131160ce74b2da74bf0eb7df138ee]) successfully deleted
I20260620 20:11:24.879648 19780 ts_tablet_manager.cc:1939] T f5764294c6b34981862995dd720ca6a5 P a5e72ce2e215407bbffef86f037239ec: Deleting consensus metadata
I20260620 20:11:24.879827 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 19474
I20260620 20:11:24.883275 19751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:24.882967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:42419 (local address 127.0.0.1:57842)
0620 20:11:24.883118 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:24.883188 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.129:42419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260620 20:11:24.883500 19265 catalog_manager.cc:4584] TS 6c71ee8496d948cb8ed6c8be20f364e7 (127.5.91.129:42419): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f5764294c6b34981862995dd720ca6a5: Network error: Client connection negotiation failed: client connection to 127.5.91.129:42419: connect: Connection refused (error 111)
I20260620 20:11:24.886669 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 19608
I20260620 20:11:24.888199 19751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:24.886879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:45065 (local address 127.0.0.1:54912)
0620 20:11:24.888053 (+ 1174us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:24.888115 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.130:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1077}
I20260620 20:11:24.891794 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 19246
I20260620 20:11:24.897572 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:46321
--webserver_interface=127.5.91.190
--webserver_port=44725
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:36163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:46321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:11:25.020821 19782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:25.021142 19782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:25.021202 19782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:25.025979 19782 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:11:25.026117 19782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:25.026185 19782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:25.026211 19782 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:11:25.026245 19782 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:11:25.033201 19782 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:36163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:46321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:46321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=44725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:25.034416 19782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:25.035435 19782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:25.050098 19791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:25.050092 19789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:25.050179 19782 server_base.cc:1034] running on GCE node
W20260620 20:11:25.050092 19788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:25.050942 19782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:25.052139 19782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:25.053501 19782 hybrid_clock.cc:648] HybridClock initialized: now 1781986285053464 us; error 72 us; skew 500 ppm
Jun 20 20:11:25 dist-test-slave-l8h4 krb5kdc[19233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986285, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:25.056370 19782 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:11:25.057335 19782 webserver.cc:460] Webserver started at http://127.5.91.190:44725/ using document root <none> and password file <none>
I20260620 20:11:25.057901 19782 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:25.057976 19782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:25.062054 19782 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260620 20:11:25.064368 19798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:25.065690 19782 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 20:11:25.065809 19782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "be988ae9ff8b4530ad335b440f95e011"
format_stamp: "Formatted at 2026-06-20 20:11:23 on dist-test-slave-l8h4"
server_key: "34f43a2f7cd4c6549f5eb1fe6d31501d"
server_key_iv: "d2c26a31c8318b7bf392c621300132e8"
server_key_version: "encryptionkey@0"
I20260620 20:11:25.066499 19782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:25.084062 19782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:25.087137 19782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:25.087307 19782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:25.094260 19782 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:46321
I20260620 20:11:25.094580 19851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:46321 every 8 connection(s)
I20260620 20:11:25.095366 19782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:11:25.100039 19852 sys_catalog.cc:263] Verifying existing consensus state
I20260620 20:11:25.101855 19852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011: Bootstrap starting.
I20260620 20:11:25.105228 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 19782
I20260620 20:11:25.105664 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:42419
--local_ip_for_outbound_sockets=127.5.91.129
--tserver_master_addrs=127.5.91.190:46321
--webserver_port=39767
--webserver_interface=127.5.91.129
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:36163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 20:11:25.120746 19852 log.cc:824] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011: Log is configured to *not* fsync() on all Append() calls
I20260620 20:11:25.129537 19852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 20:11:25.129918 19852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011: Bootstrap complete.
I20260620 20:11:25.133862 19852 raft_consensus.cc:348] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be988ae9ff8b4530ad335b440f95e011" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46321 } }
I20260620 20:11:25.134315 19852 raft_consensus.cc:729] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: be988ae9ff8b4530ad335b440f95e011, State: Initialized, Role: FOLLOWER
I20260620 20:11:25.134783 19852 consensus_queue.cc:260] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be988ae9ff8b4530ad335b440f95e011" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46321 } }
I20260620 20:11:25.134900 19852 raft_consensus.cc:388] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:11:25.134974 19852 raft_consensus.cc:482] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:11:25.135066 19852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 1 FOLLOWER]: Advancing to term 2
I20260620 20:11:25.136921 19852 raft_consensus.cc:504] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be988ae9ff8b4530ad335b440f95e011" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46321 } }
I20260620 20:11:25.137235 19852 leader_election.cc:302] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be988ae9ff8b4530ad335b440f95e011; no voters:
I20260620 20:11:25.137529 19852 leader_election.cc:288] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 20:11:25.137630 19856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 20:11:25.137797 19856 raft_consensus.cc:686] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 2 LEADER]: Becoming Leader. State: Replica: be988ae9ff8b4530ad335b440f95e011, State: Running, Role: LEADER
I20260620 20:11:25.138091 19856 consensus_queue.cc:237] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be988ae9ff8b4530ad335b440f95e011" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46321 } }
I20260620 20:11:25.138820 19852 sys_catalog.cc:564] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:11:25.139492 19858 sys_catalog.cc:455] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be988ae9ff8b4530ad335b440f95e011. Latest consensus state: current_term: 2 leader_uuid: "be988ae9ff8b4530ad335b440f95e011" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be988ae9ff8b4530ad335b440f95e011" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46321 } } }
I20260620 20:11:25.139618 19858 sys_catalog.cc:458] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [sys.catalog]: This master's current role is: LEADER
I20260620 20:11:25.139540 19857 sys_catalog.cc:455] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "be988ae9ff8b4530ad335b440f95e011" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be988ae9ff8b4530ad335b440f95e011" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46321 } } }
I20260620 20:11:25.139771 19857 sys_catalog.cc:458] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [sys.catalog]: This master's current role is: LEADER
I20260620 20:11:25.140090 19861 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:11:25.145035 19861 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:11:25.145763 19861 catalog_manager.cc:1259] Loaded cluster ID: 41d963b2cb5d467d88097c2648c3f038
I20260620 20:11:25.145823 19861 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:11:25.147257 19861 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260620 20:11:25.152022 19861 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260620 20:11:25.157868 19861 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:11:25.158702 19861 catalog_manager.cc:5879] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011: Loaded TSK: 0
I20260620 20:11:25.159931 19861 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 20:11:25.227402 19854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:25.227696 19854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:25.227763 19854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:25.232429 19854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:25.232501 19854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:25.232584 19854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:11:25.238090 19854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:36163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:42419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=39767
--tserver_master_addrs=127.5.91.190:46321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:25.239130 19854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:25.240123 19854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:25.249619 19880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:25.249800 19854 server_base.cc:1034] running on GCE node
W20260620 20:11:25.249619 19879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:25.250075 19882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:25.250500 19854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:25.251142 19854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:25.252337 19854 hybrid_clock.cc:648] HybridClock initialized: now 1781986285252309 us; error 37 us; skew 500 ppm
Jun 20 20:11:25 dist-test-slave-l8h4 krb5kdc[19233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986285, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:25.255376 19854 init.cc:377] Logged in from keytab as kudu/127.5.91.129@KRBTEST.COM (short username kudu)
I20260620 20:11:25.256319 19854 webserver.cc:460] Webserver started at http://127.5.91.129:39767/ using document root <none> and password file <none>
I20260620 20:11:25.256848 19854 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:25.256932 19854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:25.261075 19854 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 20:11:25.263389 19889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:25.264626 19854 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:11:25.264730 19854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "6c71ee8496d948cb8ed6c8be20f364e7"
format_stamp: "Formatted at 2026-06-20 20:11:23 on dist-test-slave-l8h4"
server_key: "51615fdeb0d0ec0f42a5821b8c88f576"
server_key_iv: "6e075aae5f800989a072c08fa7bb4af3"
server_key_version: "encryptionkey@0"
I20260620 20:11:25.265286 19854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:25.291920 19854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:25.295302 19854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:25.295473 19854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:25.296487 19854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:11:25.298090 19896 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 20:11:25.302502 19854 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 20:11:25.302579 19854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260620 20:11:25.302651 19854 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 20:11:25.304435 19854 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 20:11:25.304489 19854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260620 20:11:25.304817 19896 tablet_bootstrap.cc:492] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7: Bootstrap starting.
I20260620 20:11:25.314245 19854 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:42419
I20260620 20:11:25.314448 20004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:42419 every 8 connection(s)
I20260620 20:11:25.315546 19854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:11:25.322324 19896 log.cc:824] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7: Log is configured to *not* fsync() on all Append() calls
I20260620 20:11:25.323547 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 19854
I20260620 20:11:25.324389 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:45065
--local_ip_for_outbound_sockets=127.5.91.130
--tserver_master_addrs=127.5.91.190:46321
--webserver_port=46335
--webserver_interface=127.5.91.130
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:36163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 20:11:25 dist-test-slave-l8h4 krb5kdc[19233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986285, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:11:25.332231 19896 tablet_bootstrap.cc:492] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 20:11:25.332727 19896 tablet_bootstrap.cc:492] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7: Bootstrap complete.
I20260620 20:11:25.334507 19896 ts_tablet_manager.cc:1397] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7: Time spent bootstrapping tablet: real 0.030s user 0.025s sys 0.003s
I20260620 20:11:25.336048 20007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:25.318258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:57567 (local address 127.5.91.190:46321)
0620 20:11:25.318665 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:25.318692 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:25.319051 (+ 359us) server_negotiation.cc:407] Connection header received
0620 20:11:25.320214 (+ 1163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:25.320229 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:25.320649 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:25.321010 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:25.322087 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.323640 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.325257 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.326321 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.326586 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:25.329855 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:25.329864 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:25.329882 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:25.329927 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:25.331964 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:25.333131 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:25.333138 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:25.333143 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:25.333339 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:25.333781 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:25.333783 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:25.333784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:25.334207 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:25.334329 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:25.334891 (+ 562us) server_negotiation.cc:299] Negotiation successful
0620 20:11:25.335176 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":132,"threads_started":1}
I20260620 20:11:25.336048 20008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:25.318222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.129:57567)
0620 20:11:25.318829 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:25.318899 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:25.319759 (+ 860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:25.321077 (+ 1318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:25.321084 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:25.321433 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:25.321910 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:25.321925 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.323820 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.323824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:25.325109 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:25.325120 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.326588 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.326593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:25.326850 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:25.327630 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:25.327649 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:25.329482 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:25.332154 (+ 2672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:25.332163 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:25.332177 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:25.332598 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:25.333504 (+ 906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:25.333507 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:25.333509 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:25.333672 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:25.334328 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:25.334335 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:25.334568 (+ 233us) client_negotiation.cc:769] Sending connection context
0620 20:11:25.334759 (+ 191us) client_negotiation.cc:240] Negotiation successful
0620 20:11:25.335147 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":138,"threads_started":1}
I20260620 20:11:25.338675 20005 heartbeater.cc:347] Connected to a master server at 127.5.91.190:46321
I20260620 20:11:25.338328 19896 raft_consensus.cc:348] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } }
I20260620 20:11:25.338900 20005 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:25.338948 19896 raft_consensus.cc:729] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c71ee8496d948cb8ed6c8be20f364e7, State: Initialized, Role: FOLLOWER
I20260620 20:11:25.339548 19896 consensus_queue.cc:260] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } }
I20260620 20:11:25.339617 20005 heartbeater.cc:511] Master 127.5.91.190:46321 requested a full tablet report, sending...
I20260620 20:11:25.340736 19896 ts_tablet_manager.cc:1428] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260620 20:11:25.343415 19816 ts_manager.cc:194] Registered new tserver with Master: 6c71ee8496d948cb8ed6c8be20f364e7 (127.5.91.129:42419)
I20260620 20:11:25.347667 19816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:57567
I20260620 20:11:25.350827 20005 heartbeater.cc:503] Master 127.5.91.190:46321 was elected leader, sending a full tablet report...
Jun 20 20:11:25 dist-test-slave-l8h4 krb5kdc[19233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986285, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for kudu/127.5.91.129@KRBTEST.COM
I20260620 20:11:25.359118 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:25.346844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:42419 (local address 127.0.0.1:57850)
0620 20:11:25.347510 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:25.347562 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:25.347719 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:25.348822 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:25.348828 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:25.349282 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:25.349427 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:25.349436 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.350801 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.350805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:25.351733 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:25.351744 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.352479 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.352482 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:25.352591 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:25.353203 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:25.353218 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:25.354884 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:25.356824 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:25.356834 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:25.356839 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:25.357136 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:25.357428 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:25.357431 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:25.357433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:25.357527 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:25.358398 (+ 871us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:25.358404 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:25.358546 (+ 142us) client_negotiation.cc:769] Sending connection context
0620 20:11:25.358703 (+ 157us) client_negotiation.cc:240] Negotiation successful
0620 20:11:25.358911 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":98,"threads_started":1}
I20260620 20:11:25.359262 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:25.346954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57850 (local address 127.5.91.129:42419)
0620 20:11:25.348062 (+ 1108us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:25.348075 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:25.348113 (+ 38us) server_negotiation.cc:407] Connection header received
0620 20:11:25.348320 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:25.348371 (+ 51us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:25.348628 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:25.348786 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:25.349558 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.350666 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.351869 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.352362 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.352461 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:25.355200 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:25.355209 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:25.355220 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:25.355244 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:25.356712 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:25.357248 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:25.357253 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:25.357254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:25.357343 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:25.357740 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:25.357745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:25.357747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:25.358245 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:25.358417 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:25.358850 (+ 433us) server_negotiation.cc:299] Negotiation successful
0620 20:11:25.359058 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":963,"thread_start_us":122,"threads_started":1}
I20260620 20:11:25.360538 19938 tablet_service.cc:1514] Processing DeleteTablet for tablet f5764294c6b34981862995dd720ca6a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:11:24 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:57850
I20260620 20:11:25.361646 20014 tablet_replica.cc:330] stopping tablet replica
I20260620 20:11:25.361974 20014 raft_consensus.cc:2229] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:11:25.362257 20014 raft_consensus.cc:2258] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:11:25.363382 20014 ts_tablet_manager.cc:1905] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:11:25.366943 20014 ts_tablet_manager.cc:1918] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:11:25.367031 20014 log.cc:1196] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/wal/wals/f5764294c6b34981862995dd720ca6a5
I20260620 20:11:25.367247 20014 ts_tablet_manager.cc:1939] T f5764294c6b34981862995dd720ca6a5 P 6c71ee8496d948cb8ed6c8be20f364e7: Deleting consensus metadata
I20260620 20:11:25.369269 19802 catalog_manager.cc:4841] TS 6c71ee8496d948cb8ed6c8be20f364e7 (127.5.91.129:42419): tablet f5764294c6b34981862995dd720ca6a5 (table test-table [id=30e131160ce74b2da74bf0eb7df138ee]) successfully deleted
W20260620 20:11:25.451104 20010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:25.451498 20010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:25.451552 20010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:25.456713 20010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:25.456801 20010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:25.456875 20010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:11:25.463775 20010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:36163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:45065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=46335
--tserver_master_addrs=127.5.91.190:46321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:25.464867 20010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:25.465847 20010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:25.476329 20020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:25.476377 20022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:25.476729 20010 server_base.cc:1034] running on GCE node
W20260620 20:11:25.476334 20019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:25.477241 20010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:25.477893 20010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:25.479076 20010 hybrid_clock.cc:648] HybridClock initialized: now 1781986285479051 us; error 33 us; skew 500 ppm
Jun 20 20:11:25 dist-test-slave-l8h4 krb5kdc[19233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986285, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:25.482075 20010 init.cc:377] Logged in from keytab as kudu/127.5.91.130@KRBTEST.COM (short username kudu)
I20260620 20:11:25.483091 20010 webserver.cc:460] Webserver started at http://127.5.91.130:46335/ using document root <none> and password file <none>
I20260620 20:11:25.483649 20010 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:25.483728 20010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:25.488265 20010 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260620 20:11:25.490622 20029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:25.492305 20010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 20:11:25.492431 20010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "a5e72ce2e215407bbffef86f037239ec"
format_stamp: "Formatted at 2026-06-20 20:11:24 on dist-test-slave-l8h4"
server_key: "58283e8d4c8ed6e0cc0140af805bdd2b"
server_key_iv: "4fa6432d01e57dbe243ecc191b6e4c1c"
server_key_version: "encryptionkey@0"
I20260620 20:11:25.493014 20010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:25.516171 20010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:25.519263 20010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:25.519445 20010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:25.520100 20010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:11:25.521052 20010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:11:25.521113 20010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:25.521163 20010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:11:25.521179 20010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:25.533239 20010 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:45065
I20260620 20:11:25.533277 20143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:45065 every 8 connection(s)
I20260620 20:11:25.534317 20010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:11:25.543805 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 20010
I20260620 20:11:25.544209 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:37033
--local_ip_for_outbound_sockets=127.5.91.131
--tserver_master_addrs=127.5.91.190:46321
--webserver_port=44899
--webserver_interface=127.5.91.131
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:36163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 20:11:25 dist-test-slave-l8h4 krb5kdc[19233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986285, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:11:25.550952 20007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:25.536218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:50789 (local address 127.5.91.190:46321)
0620 20:11:25.536407 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:25.536411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:25.537237 (+ 826us) server_negotiation.cc:407] Connection header received
0620 20:11:25.538712 (+ 1475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:25.538714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:25.538779 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:25.538861 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:25.540633 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.541271 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.542926 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.543221 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.543321 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:25.547076 (+ 3755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:25.547081 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:25.547084 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:25.547115 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:25.548447 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:25.549135 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:25.549138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:25.549140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:25.549232 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:25.549705 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:25.549708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:25.549709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:25.549900 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:25.549987 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:25.550701 (+ 714us) server_negotiation.cc:299] Negotiation successful
0620 20:11:25.550820 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260620 20:11:25.552139 20146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:25.536576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.130:50789)
0620 20:11:25.537071 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:25.537117 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:25.538383 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:25.539126 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:25.539135 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:25.539571 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:25.540361 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:25.540404 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.541463 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.541466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:25.542745 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:25.542761 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.543417 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.543422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:25.543679 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:25.544509 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:25.544529 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:25.546859 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:25.548573 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:25.548580 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:25.548601 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:25.548974 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:25.549329 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:25.549333 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:25.549335 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:25.549527 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:25.550018 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:25.550026 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:25.550409 (+ 383us) client_negotiation.cc:769] Sending connection context
0620 20:11:25.550633 (+ 224us) client_negotiation.cc:240] Negotiation successful
0620 20:11:25.551030 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":124,"threads_started":1}
I20260620 20:11:25.553638 20144 heartbeater.cc:347] Connected to a master server at 127.5.91.190:46321
I20260620 20:11:25.553915 20144 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:25.554584 20144 heartbeater.cc:511] Master 127.5.91.190:46321 requested a full tablet report, sending...
I20260620 20:11:25.555727 19816 ts_manager.cc:194] Registered new tserver with Master: a5e72ce2e215407bbffef86f037239ec (127.5.91.130:45065)
I20260620 20:11:25.556345 19816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:50789
W20260620 20:11:25.655972 20147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:25.656561 20147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:25.656819 20147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:25.661914 20147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:25.661999 20147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:25.662086 20147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:11:25.667683 20147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:36163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:37033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=44899
--tserver_master_addrs=127.5.91.190:46321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:25.668823 20147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:25.669862 20147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:25.680016 20152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:25.680022 20155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:25.680016 20153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:25.680470 20147 server_base.cc:1034] running on GCE node
I20260620 20:11:25.681003 20147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:25.681659 20147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:25.682819 20147 hybrid_clock.cc:648] HybridClock initialized: now 1781986285682798 us; error 35 us; skew 500 ppm
Jun 20 20:11:25 dist-test-slave-l8h4 krb5kdc[19233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986285, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:25.685429 20147 init.cc:377] Logged in from keytab as kudu/127.5.91.131@KRBTEST.COM (short username kudu)
I20260620 20:11:25.686324 20147 webserver.cc:460] Webserver started at http://127.5.91.131:44899/ using document root <none> and password file <none>
I20260620 20:11:25.686829 20147 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:25.686895 20147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:25.690704 20147 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 20:11:25.692778 20162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:25.694017 20147 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 20:11:25.694178 20147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "4328eee8a6e4461881c75924c93fc75f"
format_stamp: "Formatted at 2026-06-20 20:11:24 on dist-test-slave-l8h4"
server_key: "6d42255f22252c21e8984a6dce10b814"
server_key_iv: "e292c43b3ed787405bda6c70fd1e6a6d"
server_key_version: "encryptionkey@0"
I20260620 20:11:25.694754 20147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:25.706197 20147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:25.709199 20147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:25.709369 20147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:25.709925 20147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:11:25.710723 20147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:11:25.710767 20147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:25.710806 20147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:11:25.710834 20147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:25.719324 20147 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:37033
I20260620 20:11:25.719468 20275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:37033 every 8 connection(s)
I20260620 20:11:25.720304 20147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:11:25.721421 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 20147
I20260620 20:11:25.729799 20007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:25.722346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33158 (local address 127.5.91.190:46321)
0620 20:11:25.722498 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:25.722502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:25.722960 (+ 458us) server_negotiation.cc:407] Connection header received
0620 20:11:25.723150 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:25.723154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:25.723201 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:25.723271 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:25.723993 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.724512 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.725333 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.725554 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.725642 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:25.726922 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:25.726925 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:25.726927 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:25.726948 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:25.728280 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:25.728673 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:25.728677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:25.728678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:25.728751 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:25.728997 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:25.729001 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:25.729002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:25.729165 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:25.729352 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:25.729522 (+ 170us) server_negotiation.cc:299] Negotiation successful
0620 20:11:25.729629 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Jun 20 20:11:25 dist-test-slave-l8h4 krb5kdc[19233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986285, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:11:25.735409 20279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:25.722346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:54713 (local address 127.5.91.190:46321)
0620 20:11:25.722625 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:25.722628 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:25.723783 (+ 1155us) server_negotiation.cc:407] Connection header received
0620 20:11:25.725811 (+ 2028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:25.725814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:25.725858 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:25.725934 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:25.727376 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.727871 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.728993 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.729218 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.729301 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:25.732138 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:25.732141 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:25.732143 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:25.732162 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:25.733483 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:25.733961 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:25.733963 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:25.733965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:25.734032 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:25.734375 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:25.734377 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:25.734378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:25.734537 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:25.734626 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:25.735113 (+ 487us) server_negotiation.cc:299] Negotiation successful
0620 20:11:25.735228 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":86,"server-negotiator.queue_time_us":223,"thread_start_us":59,"threads_started":1}
I20260620 20:11:25.736151 20280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:25.723054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.131:54713)
0620 20:11:25.723507 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:25.723557 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:25.725507 (+ 1950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:25.726167 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:25.726177 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:25.726557 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:25.727047 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:25.727062 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.727997 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.728001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:25.728871 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:25.728881 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.729358 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.729362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:25.729513 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:25.730227 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:25.730248 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:25.731985 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:25.733607 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:25.733613 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:25.733628 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:25.733856 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:25.734114 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:25.734117 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:25.734119 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:25.734278 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:25.734642 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:25.734649 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:25.734886 (+ 237us) client_negotiation.cc:769] Sending connection context
0620 20:11:25.735039 (+ 153us) client_negotiation.cc:240] Negotiation successful
0620 20:11:25.735360 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":122,"threads_started":1}
I20260620 20:11:25.737480 20276 heartbeater.cc:347] Connected to a master server at 127.5.91.190:46321
I20260620 20:11:25.737715 20276 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:25.738250 20276 heartbeater.cc:511] Master 127.5.91.190:46321 requested a full tablet report, sending...
I20260620 20:11:25.739336 19816 ts_manager.cc:194] Registered new tserver with Master: 4328eee8a6e4461881c75924c93fc75f (127.5.91.131:37033)
I20260620 20:11:25.739977 19816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:54713
I20260620 20:11:25.743996 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 20:11:25.748210 20279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:25.744566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33168 (local address 127.5.91.190:46321)
0620 20:11:25.744721 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:25.744726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:25.744869 (+ 143us) server_negotiation.cc:407] Connection header received
0620 20:11:25.745017 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:25.745020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:25.745124 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:25.745213 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:11:25.745449 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.745971 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.746891 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.747117 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.747203 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:25.747480 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:11:25.747851 (+ 371us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:11:25.747950 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:25.748023 (+ 73us) server_negotiation.cc:299] Negotiation successful
0620 20:11:25.748080 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260620 20:11:25.748798 19816 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:33168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:11:25.752055 19816 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:11:25.759759 19938 tablet_service.cc:1467] Processing CreateTablet for tablet 8e5b6775aea94af59574bf79549d478c (DEFAULT_TABLE table=test-table [id=62a69d6bf999426dbb13f23051e0768e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:11:25.760388 19938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e5b6775aea94af59574bf79549d478c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:11:25.763116 19896 tablet_bootstrap.cc:492] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7: Bootstrap starting.
I20260620 20:11:25.764065 19896 tablet_bootstrap.cc:654] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7: Neither blocks nor log segments found. Creating new log.
I20260620 20:11:25.764076 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:25.758842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:45065 (local address 127.0.0.1:54924)
0620 20:11:25.759398 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:25.759411 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:25.759488 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:25.760168 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:25.760171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:25.760188 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:11:25.760293 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:25.760299 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.761332 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.761335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:25.762560 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:25.762571 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.763619 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.763622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:25.763717 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:25.763770 (+ 53us) client_negotiation.cc:769] Sending connection context
0620 20:11:25.763848 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 20:11:25.763942 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477}
I20260620 20:11:25.764127 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:25.759034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54924 (local address 127.5.91.130:45065)
0620 20:11:25.759700 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:25.759710 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:25.759729 (+ 19us) server_negotiation.cc:407] Connection header received
0620 20:11:25.759805 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:25.759810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:25.760025 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:25.760185 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:11:25.760401 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.761208 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.762701 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.763506 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.763603 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:25.763751 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:25.763898 (+ 147us) server_negotiation.cc:299] Negotiation successful
0620 20:11:25.763992 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":120,"threads_started":1}
I20260620 20:11:25.764871 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:25.759117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:37033 (local address 127.0.0.1:40518)
0620 20:11:25.759647 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:25.759660 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:25.759754 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:25.760410 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:25.760412 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:25.760428 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:11:25.760501 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:25.760506 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.761816 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.761820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:25.762884 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:25.762894 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.764444 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.764449 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:25.764557 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:25.764580 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 20:11:25.764666 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 20:11:25.764736 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":85,"threads_started":1}
I20260620 20:11:25.765302 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:25.759260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40518 (local address 127.5.91.131:37033)
0620 20:11:25.759649 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:25.759661 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:25.759729 (+ 68us) server_negotiation.cc:407] Connection header received
0620 20:11:25.759848 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:25.759852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:25.760025 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:25.760184 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:11:25.760644 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.761677 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.762981 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:25.763819 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:25.763922 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:25.764055 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:25.764833 (+ 778us) server_negotiation.cc:299] Negotiation successful
0620 20:11:25.765022 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":149,"threads_started":1}
I20260620 20:11:25.766582 19896 tablet_bootstrap.cc:492] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7: No bootstrap required, opened a new log
I20260620 20:11:25.766675 19896 ts_tablet_manager.cc:1397] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260620 20:11:25.767120 19896 raft_consensus.cc:348] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } }
I20260620 20:11:25.767246 19896 raft_consensus.cc:374] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:11:25.767292 19896 raft_consensus.cc:729] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c71ee8496d948cb8ed6c8be20f364e7, State: Initialized, Role: FOLLOWER
I20260620 20:11:25.767448 19896 consensus_queue.cc:260] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } }
I20260620 20:11:25.767799 19896 ts_tablet_manager.cc:1428] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 20:11:25.767951 20078 tablet_service.cc:1467] Processing CreateTablet for tablet 8e5b6775aea94af59574bf79549d478c (DEFAULT_TABLE table=test-table [id=62a69d6bf999426dbb13f23051e0768e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:11:25.767941 20210 tablet_service.cc:1467] Processing CreateTablet for tablet 8e5b6775aea94af59574bf79549d478c (DEFAULT_TABLE table=test-table [id=62a69d6bf999426dbb13f23051e0768e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:11:25.768973 20078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e5b6775aea94af59574bf79549d478c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:11:25.769143 20210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e5b6775aea94af59574bf79549d478c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:11:25.776719 20289 tablet_bootstrap.cc:492] T 8e5b6775aea94af59574bf79549d478c P 4328eee8a6e4461881c75924c93fc75f: Bootstrap starting.
I20260620 20:11:25.778301 20290 tablet_bootstrap.cc:492] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec: Bootstrap starting.
I20260620 20:11:25.780054 20289 tablet_bootstrap.cc:654] T 8e5b6775aea94af59574bf79549d478c P 4328eee8a6e4461881c75924c93fc75f: Neither blocks nor log segments found. Creating new log.
I20260620 20:11:25.780795 20290 tablet_bootstrap.cc:654] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec: Neither blocks nor log segments found. Creating new log.
I20260620 20:11:25.781178 20289 log.cc:824] T 8e5b6775aea94af59574bf79549d478c P 4328eee8a6e4461881c75924c93fc75f: Log is configured to *not* fsync() on all Append() calls
I20260620 20:11:25.781463 20290 log.cc:824] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec: Log is configured to *not* fsync() on all Append() calls
I20260620 20:11:25.783133 20290 tablet_bootstrap.cc:492] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec: No bootstrap required, opened a new log
I20260620 20:11:25.783164 20289 tablet_bootstrap.cc:492] T 8e5b6775aea94af59574bf79549d478c P 4328eee8a6e4461881c75924c93fc75f: No bootstrap required, opened a new log
I20260620 20:11:25.783329 20290 ts_tablet_manager.cc:1397] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 20:11:25.783337 20289 ts_tablet_manager.cc:1397] T 8e5b6775aea94af59574bf79549d478c P 4328eee8a6e4461881c75924c93fc75f: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260620 20:11:25.786181 20290 raft_consensus.cc:348] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } }
I20260620 20:11:25.786428 20290 raft_consensus.cc:374] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:11:25.786465 20290 raft_consensus.cc:729] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5e72ce2e215407bbffef86f037239ec, State: Initialized, Role: FOLLOWER
I20260620 20:11:25.786307 20289 raft_consensus.cc:348] T 8e5b6775aea94af59574bf79549d478c P 4328eee8a6e4461881c75924c93fc75f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } }
I20260620 20:11:25.786547 20289 raft_consensus.cc:374] T 8e5b6775aea94af59574bf79549d478c P 4328eee8a6e4461881c75924c93fc75f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:11:25.786602 20289 raft_consensus.cc:729] T 8e5b6775aea94af59574bf79549d478c P 4328eee8a6e4461881c75924c93fc75f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4328eee8a6e4461881c75924c93fc75f, State: Initialized, Role: FOLLOWER
I20260620 20:11:25.786952 20289 consensus_queue.cc:260] T 8e5b6775aea94af59574bf79549d478c P 4328eee8a6e4461881c75924c93fc75f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } }
I20260620 20:11:25.786971 20290 consensus_queue.cc:260] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } }
I20260620 20:11:25.787609 20144 heartbeater.cc:503] Master 127.5.91.190:46321 was elected leader, sending a full tablet report...
I20260620 20:11:25.787750 20276 heartbeater.cc:503] Master 127.5.91.190:46321 was elected leader, sending a full tablet report...
I20260620 20:11:25.787889 20289 ts_tablet_manager.cc:1428] T 8e5b6775aea94af59574bf79549d478c P 4328eee8a6e4461881c75924c93fc75f: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260620 20:11:25.787897 20290 ts_tablet_manager.cc:1428] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260620 20:11:25.816809 20006 tablet.cc:2369] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 20:11:25.971416 20277 tablet.cc:2369] T 8e5b6775aea94af59574bf79549d478c P 4328eee8a6e4461881c75924c93fc75f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 20:11:26.035532 20145 tablet.cc:2369] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:11:26.039995 20293 raft_consensus.cc:482] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:11:26.040225 20293 raft_consensus.cc:504] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } }
I20260620 20:11:26.041441 20293 leader_election.cc:288] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4328eee8a6e4461881c75924c93fc75f (127.5.91.131:37033), 6c71ee8496d948cb8ed6c8be20f364e7 (127.5.91.129:42419)
I20260620 20:11:26.046207 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:26.041802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:57099 (local address 127.5.91.131:37033)
0620 20:11:26.041920 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:26.041925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:26.041946 (+ 21us) server_negotiation.cc:407] Connection header received
0620 20:11:26.042102 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:26.042105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:26.042186 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:26.042260 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:11:26.042505 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:26.043262 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:26.044855 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:26.045518 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:26.045649 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:26.045704 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:26.045958 (+ 254us) server_negotiation.cc:299] Negotiation successful
0620 20:11:26.046083 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 20:11:26.046207 20296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:26.041945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:56747 (local address 127.5.91.129:42419)
0620 20:11:26.042264 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:26.042268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:26.042281 (+ 13us) server_negotiation.cc:407] Connection header received
0620 20:11:26.042333 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:26.042336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:26.042394 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:26.042486 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:11:26.042714 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:26.043418 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:26.044892 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:26.045571 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:26.045668 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:26.045798 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:26.045957 (+ 159us) server_negotiation.cc:299] Negotiation successful
0620 20:11:26.046039 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":133,"threads_started":1}
I20260620 20:11:26.046212 20146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:26.041703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:37033 (local address 127.5.91.130:57099)
0620 20:11:26.041889 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:26.041903 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:26.041994 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:26.042285 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:26.042288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:26.042303 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:11:26.042411 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:26.042415 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:26.043382 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:26.043385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:26.044708 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:26.044719 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:26.045655 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:26.045659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:26.045815 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:26.045853 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 20:11:26.045929 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 20:11:26.046028 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":61}
I20260620 20:11:26.046219 20295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:26.041764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:42419 (local address 127.5.91.130:56747)
0620 20:11:26.042157 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:26.042171 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:26.042261 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:26.042516 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:26.042519 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:26.042536 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:11:26.042619 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:26.042623 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:26.043522 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:26.043525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:26.044780 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:26.044788 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:26.045670 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:26.045673 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:26.045825 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:26.045859 (+ 34us) client_negotiation.cc:769] Sending connection context
0620 20:11:26.045929 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 20:11:26.046022 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":79,"thread_start_us":83,"threads_started":1}
I20260620 20:11:26.047031 19958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e5b6775aea94af59574bf79549d478c" candidate_uuid: "a5e72ce2e215407bbffef86f037239ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" is_pre_election: true
I20260620 20:11:26.047199 20229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e5b6775aea94af59574bf79549d478c" candidate_uuid: "a5e72ce2e215407bbffef86f037239ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4328eee8a6e4461881c75924c93fc75f" is_pre_election: true
I20260620 20:11:26.047318 19958 raft_consensus.cc:2454] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5e72ce2e215407bbffef86f037239ec in term 0.
I20260620 20:11:26.047462 20229 raft_consensus.cc:2454] T 8e5b6775aea94af59574bf79549d478c P 4328eee8a6e4461881c75924c93fc75f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5e72ce2e215407bbffef86f037239ec in term 0.
I20260620 20:11:26.047792 20033 leader_election.cc:302] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c71ee8496d948cb8ed6c8be20f364e7, a5e72ce2e215407bbffef86f037239ec; no voters:
I20260620 20:11:26.048065 20293 raft_consensus.cc:2781] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:11:26.048141 20293 raft_consensus.cc:482] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:11:26.048192 20293 raft_consensus.cc:3037] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:11:26.049216 20293 raft_consensus.cc:504] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } }
I20260620 20:11:26.049575 20293 leader_election.cc:288] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec [CANDIDATE]: Term 1 election: Requested vote from peers 4328eee8a6e4461881c75924c93fc75f (127.5.91.131:37033), 6c71ee8496d948cb8ed6c8be20f364e7 (127.5.91.129:42419)
I20260620 20:11:26.049927 20229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e5b6775aea94af59574bf79549d478c" candidate_uuid: "a5e72ce2e215407bbffef86f037239ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4328eee8a6e4461881c75924c93fc75f"
I20260620 20:11:26.050000 19958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e5b6775aea94af59574bf79549d478c" candidate_uuid: "a5e72ce2e215407bbffef86f037239ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c71ee8496d948cb8ed6c8be20f364e7"
I20260620 20:11:26.050053 20229 raft_consensus.cc:3037] T 8e5b6775aea94af59574bf79549d478c P 4328eee8a6e4461881c75924c93fc75f [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:11:26.050118 19958 raft_consensus.cc:3037] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:11:26.051040 19958 raft_consensus.cc:2454] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5e72ce2e215407bbffef86f037239ec in term 1.
I20260620 20:11:26.051057 20229 raft_consensus.cc:2454] T 8e5b6775aea94af59574bf79549d478c P 4328eee8a6e4461881c75924c93fc75f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5e72ce2e215407bbffef86f037239ec in term 1.
I20260620 20:11:26.051297 20033 leader_election.cc:302] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c71ee8496d948cb8ed6c8be20f364e7, a5e72ce2e215407bbffef86f037239ec; no voters:
I20260620 20:11:26.051462 20293 raft_consensus.cc:2781] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:11:26.051669 20293 raft_consensus.cc:686] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec [term 1 LEADER]: Becoming Leader. State: Replica: a5e72ce2e215407bbffef86f037239ec, State: Running, Role: LEADER
I20260620 20:11:26.051991 20293 consensus_queue.cc:237] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } }
I20260620 20:11:26.055377 19814 catalog_manager.cc:5495] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec reported cstate change: term changed from 0 to 1, leader changed from <none> to a5e72ce2e215407bbffef86f037239ec (127.5.91.130). New cstate: current_term: 1 leader_uuid: "a5e72ce2e215407bbffef86f037239ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } health_report { overall_health: UNKNOWN } } }
I20260620 20:11:26.135676 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:26.132153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54926 (local address 127.5.91.130:45065)
0620 20:11:26.132278 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:26.132281 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:26.132296 (+ 15us) server_negotiation.cc:407] Connection header received
0620 20:11:26.132426 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:26.132428 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:26.132482 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:26.132584 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:11:26.132871 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:26.133404 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:26.134247 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:26.134459 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:26.134548 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:26.134836 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:11:26.135249 (+ 413us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:11:26.135348 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:26.135479 (+ 131us) server_negotiation.cc:299] Negotiation successful
0620 20:11:26.135540 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 20:11:26.141733 19958 raft_consensus.cc:1261] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Refusing update from remote peer a5e72ce2e215407bbffef86f037239ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:11:26.141737 20229 raft_consensus.cc:1261] T 8e5b6775aea94af59574bf79549d478c P 4328eee8a6e4461881c75924c93fc75f [term 1 FOLLOWER]: Refusing update from remote peer a5e72ce2e215407bbffef86f037239ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:11:26.142390 20293 consensus_queue.cc:1035] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:11:26.142573 20297 consensus_queue.cc:1035] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:11:26.156131 19815 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:33168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:11:26.156286 19815 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:33168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:11:26.158593 19815 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011: Sending DeleteTablet for 3 replicas of tablet 8e5b6775aea94af59574bf79549d478c
I20260620 20:11:26.159286 19938 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e5b6775aea94af59574bf79549d478c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:11:26 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:57850
I20260620 20:11:26.159459 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 19854
I20260620 20:11:26.159524 20078 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e5b6775aea94af59574bf79549d478c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:11:26 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:54924
I20260620 20:11:26.159948 20210 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e5b6775aea94af59574bf79549d478c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:11:26 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:40518
I20260620 20:11:26.160351 20312 tablet_replica.cc:330] T 8e5b6775aea94af59574bf79549d478c P 4328eee8a6e4461881c75924c93fc75f: stopping tablet replica
I20260620 20:11:26.160688 20312 raft_consensus.cc:2229] T 8e5b6775aea94af59574bf79549d478c P 4328eee8a6e4461881c75924c93fc75f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:11:26.160967 20311 tablet_replica.cc:330] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec: stopping tablet replica
I20260620 20:11:26.161005 20312 raft_consensus.cc:2258] T 8e5b6775aea94af59574bf79549d478c P 4328eee8a6e4461881c75924c93fc75f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:11:26.161301 20311 raft_consensus.cc:2229] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec [term 1 LEADER]: Raft consensus shutting down.
I20260620 20:11:26.161684 20311 raft_consensus.cc:2258] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:11:26.162642 20312 ts_tablet_manager.cc:1905] T 8e5b6775aea94af59574bf79549d478c P 4328eee8a6e4461881c75924c93fc75f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:11:26.163106 20311 ts_tablet_manager.cc:1905] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:11:26.165354 20312 ts_tablet_manager.cc:1918] T 8e5b6775aea94af59574bf79549d478c P 4328eee8a6e4461881c75924c93fc75f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:11:26.165465 20312 log.cc:1196] T 8e5b6775aea94af59574bf79549d478c P 4328eee8a6e4461881c75924c93fc75f: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/8e5b6775aea94af59574bf79549d478c
I20260620 20:11:26.165702 20311 ts_tablet_manager.cc:1918] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:11:26.165791 20312 ts_tablet_manager.cc:1939] T 8e5b6775aea94af59574bf79549d478c P 4328eee8a6e4461881c75924c93fc75f: Deleting consensus metadata
I20260620 20:11:26.165894 20311 log.cc:1196] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/wal/wals/8e5b6775aea94af59574bf79549d478c
I20260620 20:11:26.166250 20311 ts_tablet_manager.cc:1939] T 8e5b6775aea94af59574bf79549d478c P a5e72ce2e215407bbffef86f037239ec: Deleting consensus metadata
I20260620 20:11:26.167140 19801 catalog_manager.cc:4841] TS 4328eee8a6e4461881c75924c93fc75f (127.5.91.131:37033): tablet 8e5b6775aea94af59574bf79549d478c (table test-table [id=62a69d6bf999426dbb13f23051e0768e]) successfully deleted
I20260620 20:11:26.167341 19801 catalog_manager.cc:4841] TS a5e72ce2e215407bbffef86f037239ec (127.5.91.130:45065): tablet 8e5b6775aea94af59574bf79549d478c (table test-table [id=62a69d6bf999426dbb13f23051e0768e]) successfully deleted
W20260620 20:11:26.167666 19802 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.91.129:42419 (error 108)
I20260620 20:11:26.167935 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 20010
I20260620 20:11:26.169718 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:26.169343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:42419 (local address 127.0.0.1:57854)
0620 20:11:26.169479 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:26.169586 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.129:42419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260620 20:11:26.169875 19802 catalog_manager.cc:4584] TS 6c71ee8496d948cb8ed6c8be20f364e7 (127.5.91.129:42419): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8e5b6775aea94af59574bf79549d478c: Network error: Client connection negotiation failed: client connection to 127.5.91.129:42419: connect: Connection refused (error 111)
I20260620 20:11:26.173074 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 20147
I20260620 20:11:26.177570 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 19782
I20260620 20:11:26.182927 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:46321
--webserver_interface=127.5.91.190
--webserver_port=44725
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:36163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:46321
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:11:26.295871 20314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:26.296159 20314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:26.296200 20314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:26.300329 20314 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:11:26.300400 20314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:26.300418 20314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:26.300434 20314 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:11:26.300448 20314 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:11:26.306015 20314 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:36163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:46321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:46321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=44725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:26.307106 20314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:26.308105 20314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:26.316555 20319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:26.316596 20322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:26.316743 20320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:26.316838 20314 server_base.cc:1034] running on GCE node
I20260620 20:11:26.317479 20314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:26.318595 20314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:26.319753 20314 hybrid_clock.cc:648] HybridClock initialized: now 1781986286319734 us; error 30 us; skew 500 ppm
Jun 20 20:11:26 dist-test-slave-l8h4 krb5kdc[19233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986286, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:26.323283 20314 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:11:26.324162 20314 webserver.cc:460] Webserver started at http://127.5.91.190:44725/ using document root <none> and password file <none>
I20260620 20:11:26.324707 20314 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:26.324785 20314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:26.328647 20314 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260620 20:11:26.330884 20329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:26.332173 20314 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 20:11:26.332302 20314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "be988ae9ff8b4530ad335b440f95e011"
format_stamp: "Formatted at 2026-06-20 20:11:23 on dist-test-slave-l8h4"
server_key: "34f43a2f7cd4c6549f5eb1fe6d31501d"
server_key_iv: "d2c26a31c8318b7bf392c621300132e8"
server_key_version: "encryptionkey@0"
I20260620 20:11:26.332876 20314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:26.352396 20314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:26.356531 20314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:26.356735 20314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:26.363804 20314 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:46321
I20260620 20:11:26.364144 20383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:46321 every 8 connection(s)
I20260620 20:11:26.365095 20314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:11:26.369603 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 20314
I20260620 20:11:26.370039 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:42419
--local_ip_for_outbound_sockets=127.5.91.129
--tserver_master_addrs=127.5.91.190:46321
--webserver_port=39767
--webserver_interface=127.5.91.129
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:36163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 20:11:26.371052 20384 sys_catalog.cc:263] Verifying existing consensus state
I20260620 20:11:26.373219 20384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011: Bootstrap starting.
I20260620 20:11:26.396685 20384 log.cc:824] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011: Log is configured to *not* fsync() on all Append() calls
I20260620 20:11:26.409032 20384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 20:11:26.409408 20384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011: Bootstrap complete.
I20260620 20:11:26.413211 20384 raft_consensus.cc:348] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be988ae9ff8b4530ad335b440f95e011" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46321 } }
I20260620 20:11:26.414023 20384 raft_consensus.cc:729] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: be988ae9ff8b4530ad335b440f95e011, State: Initialized, Role: FOLLOWER
I20260620 20:11:26.414772 20384 consensus_queue.cc:260] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be988ae9ff8b4530ad335b440f95e011" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46321 } }
I20260620 20:11:26.414944 20384 raft_consensus.cc:388] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:11:26.415011 20384 raft_consensus.cc:482] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:11:26.415104 20384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 2 FOLLOWER]: Advancing to term 3
I20260620 20:11:26.417380 20384 raft_consensus.cc:504] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be988ae9ff8b4530ad335b440f95e011" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46321 } }
I20260620 20:11:26.417737 20384 leader_election.cc:302] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be988ae9ff8b4530ad335b440f95e011; no voters:
I20260620 20:11:26.418046 20384 leader_election.cc:288] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260620 20:11:26.418175 20388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 3 FOLLOWER]: Leader election won for term 3
I20260620 20:11:26.418349 20388 raft_consensus.cc:686] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [term 3 LEADER]: Becoming Leader. State: Replica: be988ae9ff8b4530ad335b440f95e011, State: Running, Role: LEADER
I20260620 20:11:26.418646 20388 consensus_queue.cc:237] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be988ae9ff8b4530ad335b440f95e011" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46321 } }
I20260620 20:11:26.419278 20384 sys_catalog.cc:564] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:11:26.419566 20390 sys_catalog.cc:455] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be988ae9ff8b4530ad335b440f95e011. Latest consensus state: current_term: 3 leader_uuid: "be988ae9ff8b4530ad335b440f95e011" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be988ae9ff8b4530ad335b440f95e011" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46321 } } }
I20260620 20:11:26.419687 20390 sys_catalog.cc:458] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [sys.catalog]: This master's current role is: LEADER
I20260620 20:11:26.419903 20389 sys_catalog.cc:455] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "be988ae9ff8b4530ad335b440f95e011" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be988ae9ff8b4530ad335b440f95e011" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46321 } } }
I20260620 20:11:26.419999 20389 sys_catalog.cc:458] T 00000000000000000000000000000000 P be988ae9ff8b4530ad335b440f95e011 [sys.catalog]: This master's current role is: LEADER
I20260620 20:11:26.420466 20392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:11:26.425616 20392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:11:26.426913 20392 catalog_manager.cc:1259] Loaded cluster ID: 41d963b2cb5d467d88097c2648c3f038
I20260620 20:11:26.426988 20392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260620 20:11:26.428148 20392 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f746f7339ad google::LogMessage::Fail() at ??:0
@ 0x7f746f73688a google::LogMessage::SendToLog() at ??:0
@ 0x7f746f7334de google::LogMessage::Flush() at ??:0
@ 0x7f746f734a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f747a072fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f747a073d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f747a070579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f747a0b4640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f74719853d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f7471985c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f74719873f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f7471976512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f7472fba6db start_thread at ??:0
@ 0x7f746e23161f clone at ??:0
W20260620 20:11:26.514014 20386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:26.514376 20386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:26.514441 20386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:26.519368 20386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:26.519440 20386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:26.519515 20386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:11:26.525306 20386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:36163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:42419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=39767
--tserver_master_addrs=127.5.91.190:46321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:26.526630 20386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:26.527645 20386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:26.539238 20428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:26.539255 20425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:26.539366 20426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:26.539410 20386 server_base.cc:1034] running on GCE node
I20260620 20:11:26.540235 20386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:26.540992 20386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:26.542223 20386 hybrid_clock.cc:648] HybridClock initialized: now 1781986286542200 us; error 35 us; skew 500 ppm
Jun 20 20:11:26 dist-test-slave-l8h4 krb5kdc[19233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986286, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:26.551458 20386 init.cc:377] Logged in from keytab as kudu/127.5.91.129@KRBTEST.COM (short username kudu)
I20260620 20:11:26.552805 20386 webserver.cc:460] Webserver started at http://127.5.91.129:39767/ using document root <none> and password file <none>
I20260620 20:11:26.553578 20386 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:26.553678 20386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:26.559610 20386 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260620 20:11:26.562318 20435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:26.563958 20386 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:11:26.564069 20386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "6c71ee8496d948cb8ed6c8be20f364e7"
format_stamp: "Formatted at 2026-06-20 20:11:23 on dist-test-slave-l8h4"
server_key: "51615fdeb0d0ec0f42a5821b8c88f576"
server_key_iv: "6e075aae5f800989a072c08fa7bb4af3"
server_key_version: "encryptionkey@0"
I20260620 20:11:26.564743 20386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:26.582707 20386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:26.587157 20386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:26.587365 20386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:26.588127 20386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:11:26.589617 20442 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 20:11:26.595302 20386 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 20:11:26.595407 20386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260620 20:11:26.595479 20386 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 20:11:26.597892 20386 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 20:11:26.597970 20386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260620 20:11:26.598425 20442 tablet_bootstrap.cc:492] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7: Bootstrap starting.
I20260620 20:11:26.609522 20386 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:42419
I20260620 20:11:26.609890 20550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:42419 every 8 connection(s)
I20260620 20:11:26.610849 20386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:11:26.618733 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 20386
I20260620 20:11:26.618935 20442 log.cc:824] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7: Log is configured to *not* fsync() on all Append() calls
I20260620 20:11:26.619195 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:45065
--local_ip_for_outbound_sockets=127.5.91.130
--tserver_master_addrs=127.5.91.190:46321
--webserver_port=46335
--webserver_interface=127.5.91.130
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:36163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 20:11:26.629086 20442 tablet_bootstrap.cc:492] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 20:11:26.629528 20442 tablet_bootstrap.cc:492] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7: Bootstrap complete.
I20260620 20:11:26.630838 20442 ts_tablet_manager.cc:1397] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7: Time spent bootstrapping tablet: real 0.033s user 0.016s sys 0.015s
I20260620 20:11:26.633963 20442 raft_consensus.cc:348] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } }
I20260620 20:11:26.634567 20442 raft_consensus.cc:729] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c71ee8496d948cb8ed6c8be20f364e7, State: Initialized, Role: FOLLOWER
I20260620 20:11:26.635102 20442 consensus_queue.cc:260] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } }
I20260620 20:11:26.636163 20442 ts_tablet_manager.cc:1428] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
W20260620 20:11:26.694043 20553 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:11:26.613171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.129:36895)
0620 20:11:26.613759 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:26.613816 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:26.615074 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:26.693283 (+ 78209us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":166,"threads_started":1}
W20260620 20:11:26.694577 20551 heartbeater.cc:644] Failed to heartbeat to 127.5.91.190:46321 (0 consecutive failures): Network error: Failed to ping master at 127.5.91.190:46321: Client connection negotiation failed: client connection to 127.5.91.190:46321: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 20:11:26.695317 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:26.695058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.129:42163)
0620 20:11:26.695194 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:26.695258 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260620 20:11:26.695967 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:26.695738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.129:56217)
0620 20:11:26.695866 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:26.695915 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260620 20:11:26.696146 20551 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 20:11:26.739152 20555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:26.739425 20555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:26.739485 20555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:26.743891 20555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:26.743974 20555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:26.744048 20555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:11:26.750406 20555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:36163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:45065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=46335
--tserver_master_addrs=127.5.91.190:46321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:26.752152 20555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:26.753356 20555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:26.763386 20564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:26.763375 20561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:26.763375 20562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:26.763779 20555 server_base.cc:1034] running on GCE node
I20260620 20:11:26.764214 20555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:26.764858 20555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:26.766031 20555 hybrid_clock.cc:648] HybridClock initialized: now 1781986286766008 us; error 33 us; skew 500 ppm
Jun 20 20:11:26 dist-test-slave-l8h4 krb5kdc[19233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986286, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:26.769032 20555 init.cc:377] Logged in from keytab as kudu/127.5.91.130@KRBTEST.COM (short username kudu)
I20260620 20:11:26.770299 20555 webserver.cc:460] Webserver started at http://127.5.91.130:46335/ using document root <none> and password file <none>
I20260620 20:11:26.770872 20555 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:26.770965 20555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:26.774914 20555 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260620 20:11:26.776990 20571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:26.778375 20555 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:11:26.778483 20555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "a5e72ce2e215407bbffef86f037239ec"
format_stamp: "Formatted at 2026-06-20 20:11:24 on dist-test-slave-l8h4"
server_key: "58283e8d4c8ed6e0cc0140af805bdd2b"
server_key_iv: "4fa6432d01e57dbe243ecc191b6e4c1c"
server_key_version: "encryptionkey@0"
I20260620 20:11:26.779042 20555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:26.801203 20555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:26.804273 20555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:26.804445 20555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:26.805017 20555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:11:26.805807 20555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:11:26.805850 20555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:26.805889 20555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:11:26.805917 20555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:26.815088 20555 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:45065
I20260620 20:11:26.815168 20684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:45065 every 8 connection(s)
I20260620 20:11:26.816136 20555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:11:26.817127 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 20555
I20260620 20:11:26.817512 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:37033
--local_ip_for_outbound_sockets=127.5.91.131
--tserver_master_addrs=127.5.91.190:46321
--webserver_port=44899
--webserver_interface=127.5.91.131
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:36163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 20:11:26.820823 20687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:26.818596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.130:40655)
0620 20:11:26.819238 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:26.819381 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":150,"threads_started":1}
W20260620 20:11:26.821307 20685 heartbeater.cc:644] Failed to heartbeat to 127.5.91.190:46321 (0 consecutive failures): Network error: Failed to ping master at 127.5.91.190:46321: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
I20260620 20:11:26.821949 20687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:26.821701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.130:46783)
0620 20:11:26.821825 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:26.821881 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260620 20:11:26.822808 20687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:26.822565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.130:52597)
0620 20:11:26.822681 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:26.822736 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260620 20:11:26.823028 20685 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 20:11:26.937544 20688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:26.938045 20688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:26.938107 20688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:26.942672 20688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:26.942745 20688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:26.942821 20688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:11:26.948508 20688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:36163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:37033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=44899
--tserver_master_addrs=127.5.91.190:46321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:26.949622 20688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:26.950654 20688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:26.960208 20693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:26.960227 20696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:26.960201 20694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:26.960434 20688 server_base.cc:1034] running on GCE node
I20260620 20:11:26.960922 20688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:26.961527 20688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:26.962764 20688 hybrid_clock.cc:648] HybridClock initialized: now 1781986286962735 us; error 38 us; skew 500 ppm
Jun 20 20:11:26 dist-test-slave-l8h4 krb5kdc[19233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986286, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:26.965718 20688 init.cc:377] Logged in from keytab as kudu/127.5.91.131@KRBTEST.COM (short username kudu)
I20260620 20:11:26.966642 20688 webserver.cc:460] Webserver started at http://127.5.91.131:44899/ using document root <none> and password file <none>
I20260620 20:11:26.967160 20688 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:26.967217 20688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:26.971032 20688 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:11:26.973259 20703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:26.974663 20688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:11:26.974771 20688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "4328eee8a6e4461881c75924c93fc75f"
format_stamp: "Formatted at 2026-06-20 20:11:24 on dist-test-slave-l8h4"
server_key: "6d42255f22252c21e8984a6dce10b814"
server_key_iv: "e292c43b3ed787405bda6c70fd1e6a6d"
server_key_version: "encryptionkey@0"
I20260620 20:11:26.975358 20688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:26.988680 20688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:26.991737 20688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:26.991897 20688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:26.992486 20688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:11:26.993294 20688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:11:26.993337 20688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:26.993378 20688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:11:26.993409 20688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:27.002725 20688 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:37033
I20260620 20:11:27.002812 20817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:37033 every 8 connection(s)
I20260620 20:11:27.003749 20688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:11:27.005977 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 20688
I20260620 20:11:27.010223 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:27.007045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.131:48143)
0620 20:11:27.008225 (+ 1180us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:27.008360 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":903,"thread_start_us":712,"threads_started":1}
W20260620 20:11:27.010928 20818 heartbeater.cc:644] Failed to heartbeat to 127.5.91.190:46321 (0 consecutive failures): Network error: Failed to ping master at 127.5.91.190:46321: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
I20260620 20:11:27.012333 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:27.011928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.131:59341)
0620 20:11:27.012105 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:27.012173 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260620 20:11:27.014125 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:27.013565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.131:50279)
0620 20:11:27.013980 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:27.014061 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260620 20:11:27.014544 20818 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 20:11:27.697146 20822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:27.696689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.129:38871)
0620 20:11:27.696981 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:27.697038 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":75,"threads_started":1}
I20260620 20:11:27.824076 20823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:27.823614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.130:41355)
0620 20:11:27.823870 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:27.823977 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":80,"threads_started":1}
I20260620 20:11:28.016072 20824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:28.015503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.131:45687)
0620 20:11:28.015850 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:28.015936 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":115,"threads_started":1}
I20260620 20:11:28.098711 20825 raft_consensus.cc:482] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:11:28.098899 20825 raft_consensus.cc:504] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } }
I20260620 20:11:28.100054 20825 leader_election.cc:288] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a5e72ce2e215407bbffef86f037239ec (127.5.91.130:45065), 4328eee8a6e4461881c75924c93fc75f (127.5.91.131:37033)
Jun 20 20:11:28 dist-test-slave-l8h4 krb5kdc[19233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986286, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.131@KRBTEST.COM
Jun 20 20:11:28 dist-test-slave-l8h4 krb5kdc[19233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986286, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.130@KRBTEST.COM
I20260620 20:11:28.113665 20827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:28.100587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:37033 (local address 127.5.91.129:46011)
0620 20:11:28.100884 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:28.100897 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:28.100980 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:28.103150 (+ 2170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:28.103153 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:28.103403 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:28.103588 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:28.103603 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:28.105051 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:28.105054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:28.105868 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:28.105877 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:28.106611 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:28.106615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:28.106775 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:28.107330 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:28.107352 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:28.108916 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:28.111036 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:28.111042 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:28.111056 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:28.111416 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:28.111735 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:28.111738 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:28.111739 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:28.111912 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:28.112813 (+ 901us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:28.112818 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:28.113017 (+ 199us) client_negotiation.cc:769] Sending connection context
0620 20:11:28.113183 (+ 166us) client_negotiation.cc:240] Negotiation successful
0620 20:11:28.113444 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":43,"spinlock_wait_cycles":28672,"thread_start_us":70,"threads_started":1}
I20260620 20:11:28.113799 20828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:28.100783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:46011 (local address 127.5.91.131:37033)
0620 20:11:28.101140 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:28.101158 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:28.101186 (+ 28us) server_negotiation.cc:407] Connection header received
0620 20:11:28.101906 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:28.101913 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:28.102369 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:28.102689 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:28.103756 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:28.104895 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:28.105983 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:28.106490 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:28.106649 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:28.109136 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:28.109142 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:28.109153 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:28.109181 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:28.110920 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:28.111520 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:28.111524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:28.111528 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:28.111639 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:28.112239 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:28.112243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:28.112245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:28.112657 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:28.112754 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:28.113348 (+ 594us) server_negotiation.cc:299] Negotiation successful
0620 20:11:28.113618 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":78,"threads_started":1}
I20260620 20:11:28.114104 20826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:28.100493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:40489 (local address 127.5.91.130:45065)
0620 20:11:28.100948 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:28.100972 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:28.101016 (+ 44us) server_negotiation.cc:407] Connection header received
0620 20:11:28.101896 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:28.101906 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:28.102369 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:28.102689 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:28.103771 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:28.104800 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:28.105986 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:28.106587 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:28.106810 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:28.109299 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:28.109304 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:28.109314 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:28.109342 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:28.110949 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:28.111513 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:28.111518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:28.111521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:28.111654 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:28.112793 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:28.112796 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:28.112797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:28.113268 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:28.113412 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:28.113722 (+ 310us) server_negotiation.cc:299] Negotiation successful
0620 20:11:28.113977 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":153,"threads_started":1}
I20260620 20:11:28.115145 20822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:28.100312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:45065 (local address 127.5.91.129:40489)
0620 20:11:28.100487 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:28.100504 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:28.100640 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:28.102773 (+ 2133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:28.102779 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:28.103150 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:28.103587 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:28.103603 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:28.104961 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:28.104964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:28.105869 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:28.105878 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:28.106780 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:28.106784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:28.106865 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:28.107467 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:28.107480 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:28.109139 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:28.111052 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:28.111055 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:28.111057 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:28.111407 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:28.111771 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:28.111774 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:28.111775 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:28.111912 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:28.113409 (+ 1497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:28.113412 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:28.113492 (+ 80us) client_negotiation.cc:769] Sending connection context
0620 20:11:28.113592 (+ 100us) client_negotiation.cc:240] Negotiation successful
0620 20:11:28.113708 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"mutex_wait_us":44,"spinlock_wait_cycles":1920}
W20260620 20:11:28.116354 20438 leader_election.cc:341] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4328eee8a6e4461881c75924c93fc75f (127.5.91.131:37033): Not found: Tablet not found: 8e5b6775aea94af59574bf79549d478c
W20260620 20:11:28.116812 20438 leader_election.cc:341] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a5e72ce2e215407bbffef86f037239ec (127.5.91.130:45065): Not found: Tablet not found: 8e5b6775aea94af59574bf79549d478c
I20260620 20:11:28.116891 20438 leader_election.cc:302] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c71ee8496d948cb8ed6c8be20f364e7; no voters: 4328eee8a6e4461881c75924c93fc75f, a5e72ce2e215407bbffef86f037239ec
I20260620 20:11:28.117245 20825 raft_consensus.cc:2726] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 20:11:28.698465 20829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:28.697966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.129:45943)
0620 20:11:28.698300 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:28.698359 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":84,"threads_started":1}
I20260620 20:11:28.825297 20830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:28.824894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.130:35297)
0620 20:11:28.825153 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:28.825206 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":76,"threads_started":1}
I20260620 20:11:29.017517 20831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:29.017062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.131:46351)
0620 20:11:29.017342 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:29.017404 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":79,"threads_started":1}
I20260620 20:11:29.699749 20832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:29.699274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.129:40975)
0620 20:11:29.699590 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:29.699652 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":79,"threads_started":1}
I20260620 20:11:29.826474 20833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:29.826002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.130:52727)
0620 20:11:29.826292 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:29.826359 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":88,"threads_started":1}
I20260620 20:11:29.993528 20834 raft_consensus.cc:482] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:11:29.993736 20834 raft_consensus.cc:504] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } }
I20260620 20:11:29.994428 20834 leader_election.cc:288] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a5e72ce2e215407bbffef86f037239ec (127.5.91.130:45065), 4328eee8a6e4461881c75924c93fc75f (127.5.91.131:37033)
W20260620 20:11:29.995335 20438 leader_election.cc:341] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a5e72ce2e215407bbffef86f037239ec (127.5.91.130:45065): Not found: Tablet not found: 8e5b6775aea94af59574bf79549d478c
W20260620 20:11:29.995523 20438 leader_election.cc:341] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4328eee8a6e4461881c75924c93fc75f (127.5.91.131:37033): Not found: Tablet not found: 8e5b6775aea94af59574bf79549d478c
I20260620 20:11:29.995571 20438 leader_election.cc:302] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c71ee8496d948cb8ed6c8be20f364e7; no voters: 4328eee8a6e4461881c75924c93fc75f, a5e72ce2e215407bbffef86f037239ec
I20260620 20:11:29.995749 20834 raft_consensus.cc:2726] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 20:11:30.019040 20835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:30.018578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.131:48903)
0620 20:11:30.018869 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:30.018935 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":76,"threads_started":1}
I20260620 20:11:30.701212 20836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:30.700754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.129:49597)
0620 20:11:30.701046 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:30.701100 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":80,"threads_started":1}
I20260620 20:11:30.827696 20837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:30.827250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.130:40381)
0620 20:11:30.827542 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:30.827601 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":77,"threads_started":1}
I20260620 20:11:31.020412 20838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:31.020002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.131:59699)
0620 20:11:31.020247 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:31.020303 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":80,"threads_started":1}
I20260620 20:11:31.702520 20839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:31.702052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.129:52931)
0620 20:11:31.702364 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:31.702420 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":104,"threads_started":1}
I20260620 20:11:31.829164 20840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:31.828506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.130:43465)
0620 20:11:31.828810 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:31.829009 (+ 199us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":96,"threads_started":1}
I20260620 20:11:32.021989 20841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:32.021300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.131:47471)
0620 20:11:32.021627 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:32.021793 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":103,"threads_started":1}
I20260620 20:11:32.177767 20842 raft_consensus.cc:482] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:11:32.177908 20842 raft_consensus.cc:504] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } }
I20260620 20:11:32.178367 20842 leader_election.cc:288] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a5e72ce2e215407bbffef86f037239ec (127.5.91.130:45065), 4328eee8a6e4461881c75924c93fc75f (127.5.91.131:37033)
W20260620 20:11:32.179111 20438 leader_election.cc:341] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a5e72ce2e215407bbffef86f037239ec (127.5.91.130:45065): Not found: Tablet not found: 8e5b6775aea94af59574bf79549d478c
W20260620 20:11:32.179252 20438 leader_election.cc:341] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4328eee8a6e4461881c75924c93fc75f (127.5.91.131:37033): Not found: Tablet not found: 8e5b6775aea94af59574bf79549d478c
I20260620 20:11:32.179302 20438 leader_election.cc:302] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c71ee8496d948cb8ed6c8be20f364e7; no voters: 4328eee8a6e4461881c75924c93fc75f, a5e72ce2e215407bbffef86f037239ec
I20260620 20:11:32.179440 20842 raft_consensus.cc:2726] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 20:11:32.703835 20843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:32.703366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.129:56193)
0620 20:11:32.703661 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:32.703731 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":74,"threads_started":1}
I20260620 20:11:32.830487 20844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:32.830004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.130:43823)
0620 20:11:32.830305 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:32.830387 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":79,"threads_started":1}
I20260620 20:11:33.023334 20845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:33.022864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.131:54359)
0620 20:11:33.023173 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:33.023237 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":86,"threads_started":1}
I20260620 20:11:33.705087 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:33.704566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.129:52511)
0620 20:11:33.704894 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:33.704952 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":89,"threads_started":1}
I20260620 20:11:33.799182 20847 raft_consensus.cc:482] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:11:33.799321 20847 raft_consensus.cc:504] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } }
I20260620 20:11:33.799700 20847 leader_election.cc:288] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a5e72ce2e215407bbffef86f037239ec (127.5.91.130:45065), 4328eee8a6e4461881c75924c93fc75f (127.5.91.131:37033)
W20260620 20:11:33.800523 20438 leader_election.cc:341] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4328eee8a6e4461881c75924c93fc75f (127.5.91.131:37033): Not found: Tablet not found: 8e5b6775aea94af59574bf79549d478c
W20260620 20:11:33.800652 20438 leader_election.cc:341] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a5e72ce2e215407bbffef86f037239ec (127.5.91.130:45065): Not found: Tablet not found: 8e5b6775aea94af59574bf79549d478c
I20260620 20:11:33.800688 20438 leader_election.cc:302] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c71ee8496d948cb8ed6c8be20f364e7; no voters: 4328eee8a6e4461881c75924c93fc75f, a5e72ce2e215407bbffef86f037239ec
I20260620 20:11:33.800817 20847 raft_consensus.cc:2726] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 20:11:33.831849 20848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:33.831362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.130:50565)
0620 20:11:33.831691 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:33.831754 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":79,"threads_started":1}
I20260620 20:11:34.024726 20849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:34.024206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.131:44583)
0620 20:11:34.024545 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:34.024614 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":78,"threads_started":1}
I20260620 20:11:34.706501 20850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:34.705870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.129:32777)
0620 20:11:34.706339 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:34.706402 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":92,"threads_started":1}
I20260620 20:11:34.833149 20851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:34.832633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.130:41833)
0620 20:11:34.832920 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:34.833035 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":73,"threads_started":1}
I20260620 20:11:35.025995 20852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:35.025562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.131:60247)
0620 20:11:35.025838 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:35.025899 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":93,"threads_started":1}
I20260620 20:11:35.708215 20853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:35.707594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.129:35603)
0620 20:11:35.708025 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:35.708108 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":123,"threads_started":1}
I20260620 20:11:35.834470 20854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:35.834016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.130:41607)
0620 20:11:35.834310 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:35.834374 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":74,"threads_started":1}
I20260620 20:11:36.027372 20855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:36.026901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.131:57429)
0620 20:11:36.027205 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:36.027269 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":104,"threads_started":1}
I20260620 20:11:36.709700 20856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:36.709250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.129:53279)
0620 20:11:36.709541 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:36.709603 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":82,"threads_started":1}
I20260620 20:11:36.835675 20857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:36.835238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.130:45735)
0620 20:11:36.835518 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:36.835578 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":74,"threads_started":1}
I20260620 20:11:37.028841 20858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:37.028376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.131:45013)
0620 20:11:37.028680 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:37.028743 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":86,"threads_started":1}
I20260620 20:11:37.711442 20859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:37.710844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.129:59251)
0620 20:11:37.711215 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:37.711307 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":85,"threads_started":1}
I20260620 20:11:37.837404 20860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:37.836756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.130:53667)
0620 20:11:37.837196 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:37.837286 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":101,"threads_started":1}
I20260620 20:11:38.030608 20861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:38.029933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.131:58647)
0620 20:11:38.030375 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:38.030470 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":95,"threads_started":1}
I20260620 20:11:38.712745 20862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:38.712314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.129:60873)
0620 20:11:38.712586 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:38.712646 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":81,"threads_started":1}
I20260620 20:11:38.838796 20864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:38.838296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.130:45173)
0620 20:11:38.838564 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:38.838695 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":91,"threads_started":1}
I20260620 20:11:39.031929 20865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:39.031496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.131:47525)
0620 20:11:39.031770 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:39.031829 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":77,"threads_started":1}
I20260620 20:11:39.663334 20866 raft_consensus.cc:482] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:11:39.663529 20866 raft_consensus.cc:504] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } }
I20260620 20:11:39.664014 20866 leader_election.cc:288] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a5e72ce2e215407bbffef86f037239ec (127.5.91.130:45065), 4328eee8a6e4461881c75924c93fc75f (127.5.91.131:37033)
W20260620 20:11:39.665176 20438 leader_election.cc:341] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a5e72ce2e215407bbffef86f037239ec (127.5.91.130:45065): Not found: Tablet not found: 8e5b6775aea94af59574bf79549d478c
W20260620 20:11:39.665369 20438 leader_election.cc:341] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4328eee8a6e4461881c75924c93fc75f (127.5.91.131:37033): Not found: Tablet not found: 8e5b6775aea94af59574bf79549d478c
I20260620 20:11:39.665421 20438 leader_election.cc:302] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c71ee8496d948cb8ed6c8be20f364e7; no voters: 4328eee8a6e4461881c75924c93fc75f, a5e72ce2e215407bbffef86f037239ec
I20260620 20:11:39.665680 20866 raft_consensus.cc:2726] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 20:11:39.714421 20867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:39.713772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.129:46963)
0620 20:11:39.714191 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:39.714290 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":108,"threads_started":1}
I20260620 20:11:39.840265 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:39.839734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.130:52107)
0620 20:11:39.840048 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:39.840134 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":80,"threads_started":1}
I20260620 20:11:40.033272 20869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:40.032697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.131:35571)
0620 20:11:40.033050 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:40.033122 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":102,"threads_started":1}
I20260620 20:11:40.715678 20870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:40.715213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.129:51533)
0620 20:11:40.715526 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:40.715583 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":113,"threads_started":1}
I20260620 20:11:40.841627 20871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:40.841162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.130:47865)
0620 20:11:40.841460 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:40.841526 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":89,"threads_started":1}
I20260620 20:11:41.034534 20872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:41.034076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.131:36575)
0620 20:11:41.034375 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:41.034432 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":112,"threads_started":1}
I20260620 20:11:41.506891 20873 raft_consensus.cc:482] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:11:41.507021 20873 raft_consensus.cc:504] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e72ce2e215407bbffef86f037239ec" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 45065 } } peers { permanent_uuid: "4328eee8a6e4461881c75924c93fc75f" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 37033 } } peers { permanent_uuid: "6c71ee8496d948cb8ed6c8be20f364e7" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 42419 } }
I20260620 20:11:41.507390 20873 leader_election.cc:288] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a5e72ce2e215407bbffef86f037239ec (127.5.91.130:45065), 4328eee8a6e4461881c75924c93fc75f (127.5.91.131:37033)
W20260620 20:11:41.508191 20438 leader_election.cc:341] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4328eee8a6e4461881c75924c93fc75f (127.5.91.131:37033): Not found: Tablet not found: 8e5b6775aea94af59574bf79549d478c
W20260620 20:11:41.508329 20438 leader_election.cc:341] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a5e72ce2e215407bbffef86f037239ec (127.5.91.130:45065): Not found: Tablet not found: 8e5b6775aea94af59574bf79549d478c
I20260620 20:11:41.508383 20438 leader_election.cc:302] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c71ee8496d948cb8ed6c8be20f364e7; no voters: 4328eee8a6e4461881c75924c93fc75f, a5e72ce2e215407bbffef86f037239ec
I20260620 20:11:41.508556 20873 raft_consensus.cc:2726] T 8e5b6775aea94af59574bf79549d478c P 6c71ee8496d948cb8ed6c8be20f364e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 20:11:41.717006 20874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:41.716608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.129:37575)
0620 20:11:41.716852 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:41.716904 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":68,"threads_started":1}
I20260620 20:11:41.843020 20875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:41.842607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46321 (local address 127.5.91.130:42595)
0620 20:11:41.842858 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:41.842916 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":71,"threads_started":1}
I20260620 20:11:42.009435 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 20386
I20260620 20:11:42.014688 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 20555
I20260620 20:11:42.019312 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 20688
2026-06-20T20:11:42Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19626 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:11:42 dist-test-slave-l8h4 krb5kdc[20878](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:11:42 dist-test-slave-l8h4 krb5kdc[20878](info): set up 2 sockets
Jun 20 20:11:42 dist-test-slave-l8h4 krb5kdc[20878](info): commencing operation
krb5kdc: starting...
W20260620 20:11:42.983058 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.942s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:11:43 dist-test-slave-l8h4 krb5kdc[20878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:11:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:11:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:11:43.105509 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:46243
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:46243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:11:43.215476 20891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:43.215759 20891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:43.215832 20891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:43.220331 20891 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 20:11:43.220407 20891 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 20:11:43.220431 20891 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 20:11:43.220449 20891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:11:43.220474 20891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:43.220500 20891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:43.220534 20891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:11:43.220563 20891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:11:43.226296 20891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:46243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:46243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:43.227454 20891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:43.228474 20891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 20:11:43.237244 20891 server_base.cc:1034] running on GCE node
W20260620 20:11:43.237310 20897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:43.237409 20899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:43.237310 20896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:43.238101 20891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:43.239162 20891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:43.240329 20891 hybrid_clock.cc:648] HybridClock initialized: now 1781986303240308 us; error 32 us; skew 500 ppm
Jun 20 20:11:43 dist-test-slave-l8h4 krb5kdc[20878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986303, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:43.243330 20891 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:11:43.244210 20891 webserver.cc:460] Webserver started at http://127.5.91.190:37723/ using document root <none> and password file <none>
I20260620 20:11:43.244747 20891 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:43.244808 20891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:43.244992 20891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:11:43.247009 20891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "f6478905ca1f4e50b944e5274c384f90"
format_stamp: "Formatted at 2026-06-20 20:11:43 on dist-test-slave-l8h4"
server_key: "e71e575c547b495726c25ec2bd78000d"
server_key_iv: "4bdbda48ef9ac917c0e28347a60d7587"
server_key_version: "encryptionkey@0"
I20260620 20:11:43.247534 20891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "f6478905ca1f4e50b944e5274c384f90"
format_stamp: "Formatted at 2026-06-20 20:11:43 on dist-test-slave-l8h4"
server_key: "e71e575c547b495726c25ec2bd78000d"
server_key_iv: "4bdbda48ef9ac917c0e28347a60d7587"
server_key_version: "encryptionkey@0"
I20260620 20:11:43.251349 20891 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260620 20:11:43.253984 20906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:43.255299 20891 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:11:43.255426 20891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "f6478905ca1f4e50b944e5274c384f90"
format_stamp: "Formatted at 2026-06-20 20:11:43 on dist-test-slave-l8h4"
server_key: "e71e575c547b495726c25ec2bd78000d"
server_key_iv: "4bdbda48ef9ac917c0e28347a60d7587"
server_key_version: "encryptionkey@0"
I20260620 20:11:43.255561 20891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:43.266817 20891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:43.269711 20891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:43.269894 20891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:43.277437 20891 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:46243
I20260620 20:11:43.277725 20958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:46243 every 8 connection(s)
I20260620 20:11:43.278519 20891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:11:43.281591 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 20891
I20260620 20:11:43.281713 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:11:43.281960 20959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:11:43.281989 5486 external_mini_cluster.cc:1428] Setting key cd347d767e51637d0ce874e897522a27
I20260620 20:11:43.288691 20959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90: Bootstrap starting.
Jun 20 20:11:43 dist-test-slave-l8h4 krb5kdc[20878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:11:43.291208 20959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90: Neither blocks nor log segments found. Creating new log.
I20260620 20:11:43.292021 20959 log.cc:824] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90: Log is configured to *not* fsync() on all Append() calls
I20260620 20:11:43.294718 20959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90: No bootstrap required, opened a new log
I20260620 20:11:43.297217 20962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:43.283307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36428 (local address 127.5.91.190:46243)
0620 20:11:43.283715 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:43.283729 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:43.283763 (+ 34us) server_negotiation.cc:407] Connection header received
0620 20:11:43.284553 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:43.284571 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:43.284881 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:43.285143 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:43.286002 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:43.287202 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:43.288340 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:43.288840 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:43.288993 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:43.291552 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:43.291558 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:43.291573 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:43.291604 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:43.293934 (+ 2330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:43.294525 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:43.294530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:43.294537 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:43.294674 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:43.294963 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:43.294966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:43.294967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:43.295464 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:43.295668 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:43.295959 (+ 291us) server_negotiation.cc:299] Negotiation successful
0620 20:11:43.296255 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":274,"thread_start_us":127,"threads_started":1}
I20260620 20:11:43.299103 20959 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6478905ca1f4e50b944e5274c384f90" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46243 } }
I20260620 20:11:43.299367 20959 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:11:43.299463 20959 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6478905ca1f4e50b944e5274c384f90, State: Initialized, Role: FOLLOWER
I20260620 20:11:43.299883 20959 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6478905ca1f4e50b944e5274c384f90" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46243 } }
I20260620 20:11:43.300014 20959 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:11:43.300078 20959 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:11:43.300191 20959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:11:43.301182 20959 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6478905ca1f4e50b944e5274c384f90" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46243 } }
I20260620 20:11:43.301512 20959 leader_election.cc:302] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6478905ca1f4e50b944e5274c384f90; no voters:
I20260620 20:11:43.301816 20959 leader_election.cc:288] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:11:43.301939 20964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:11:43.302170 20964 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 1 LEADER]: Becoming Leader. State: Replica: f6478905ca1f4e50b944e5274c384f90, State: Running, Role: LEADER
I20260620 20:11:43.302505 20964 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6478905ca1f4e50b944e5274c384f90" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46243 } }
I20260620 20:11:43.303092 20959 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:11:43.304032 20965 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6478905ca1f4e50b944e5274c384f90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6478905ca1f4e50b944e5274c384f90" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46243 } } }
I20260620 20:11:43.304186 20965 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [sys.catalog]: This master's current role is: LEADER
I20260620 20:11:43.304016 20966 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6478905ca1f4e50b944e5274c384f90. Latest consensus state: current_term: 1 leader_uuid: "f6478905ca1f4e50b944e5274c384f90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6478905ca1f4e50b944e5274c384f90" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46243 } } }
I20260620 20:11:43.304390 20966 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [sys.catalog]: This master's current role is: LEADER
W20260620 20:11:43.308007 20980 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 20:11:43.308073 20980 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 20:11:43.308131 20973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:11:43.309036 20973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:11:43.314239 20973 catalog_manager.cc:1347] Generated new cluster ID: 6f3b3f9eeb994d8384ed0ab5c21539d8
I20260620 20:11:43.314301 20973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:11:43.326593 20973 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:11:43.327570 20973 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:11:43.335485 20973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90: Generated new TSK 0
I20260620 20:11:43.336341 20973 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.91.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
I20260620 20:11:43.364885 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46243
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:11:43.479031 20985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:43.479300 20985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:43.479347 20985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:43.483448 20985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:43.483515 20985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:43.483578 20985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:11:43.489192 20985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:43.490618 20985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:43.491586 20985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:43.500458 20991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:43.500514 20990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:43.500516 20993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:43.500941 20985 server_base.cc:1034] running on GCE node
I20260620 20:11:43.501308 20985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:43.501899 20985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:43.503067 20985 hybrid_clock.cc:648] HybridClock initialized: now 1781986303503053 us; error 29 us; skew 500 ppm
Jun 20 20:11:43 dist-test-slave-l8h4 krb5kdc[20878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986303, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:43.505862 20985 init.cc:377] Logged in from keytab as kudu/127.5.91.129@KRBTEST.COM (short username kudu)
I20260620 20:11:43.506776 20985 webserver.cc:460] Webserver started at http://127.5.91.129:36389/ using document root <none> and password file <none>
I20260620 20:11:43.507289 20985 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:43.507347 20985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:43.507536 20985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:11:43.509454 20985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "8d92933d8838494bb957c61d72325363"
format_stamp: "Formatted at 2026-06-20 20:11:43 on dist-test-slave-l8h4"
server_key: "fe303842c202b2d9889ca90b71a35879"
server_key_iv: "37e16f5aaf9372e3e7c3830a3901a78e"
server_key_version: "encryptionkey@0"
I20260620 20:11:43.509969 20985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "8d92933d8838494bb957c61d72325363"
format_stamp: "Formatted at 2026-06-20 20:11:43 on dist-test-slave-l8h4"
server_key: "fe303842c202b2d9889ca90b71a35879"
server_key_iv: "37e16f5aaf9372e3e7c3830a3901a78e"
server_key_version: "encryptionkey@0"
I20260620 20:11:43.513784 20985 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260620 20:11:43.516242 21000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:43.517936 20985 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 20:11:43.518074 20985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "8d92933d8838494bb957c61d72325363"
format_stamp: "Formatted at 2026-06-20 20:11:43 on dist-test-slave-l8h4"
server_key: "fe303842c202b2d9889ca90b71a35879"
server_key_iv: "37e16f5aaf9372e3e7c3830a3901a78e"
server_key_version: "encryptionkey@0"
I20260620 20:11:43.518216 20985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:43.544942 20985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:43.548131 20985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:43.548313 20985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:43.548902 20985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:11:43.549695 20985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:11:43.549762 20985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:43.549815 20985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:11:43.549842 20985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:43.560175 20985 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:38321
I20260620 20:11:43.560207 21114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:38321 every 8 connection(s)
I20260620 20:11:43.561056 20985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:11:43.561447 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 20985
I20260620 20:11:43.561574 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:11:43.561868 5486 external_mini_cluster.cc:1428] Setting key d41a1268e82898f3a2b683215b897253
WARNING: no policy specified for kudu/127.5.91.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.130@KRBTEST.COM" created.
Jun 20 20:11:43 dist-test-slave-l8h4 krb5kdc[20878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986303, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:11:43.578652 20962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:43.563064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:49647 (local address 127.5.91.190:46243)
0620 20:11:43.563171 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:43.563176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:43.564119 (+ 943us) server_negotiation.cc:407] Connection header received
0620 20:11:43.565716 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:43.565721 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:43.565777 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:43.565867 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:43.567069 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:43.567882 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:43.569048 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:43.569404 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:43.569512 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:43.574985 (+ 5473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:43.574989 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:43.574992 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:43.575022 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:43.576370 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:43.576944 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:43.576947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:43.576948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:43.577064 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:43.577445 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:43.577448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:43.577450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:43.577645 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:43.577753 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:43.578407 (+ 654us) server_negotiation.cc:299] Negotiation successful
0620 20:11:43.578523 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 20:11:43.579854 21117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:43.563474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.129:49647)
0620 20:11:43.563948 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:43.564007 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:43.565375 (+ 1368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:43.565999 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:43.566042 (+ 43us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:43.566456 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:43.566924 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:43.566937 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:43.568065 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:43.568068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:43.568925 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:43.568935 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:43.569532 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:43.569535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:43.569670 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:43.570304 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:43.570319 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:43.574791 (+ 4472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:43.576486 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:43.576492 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:43.576507 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:43.576827 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:43.577154 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:43.577157 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:43.577159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:43.577328 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:43.577765 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:43.577772 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:43.578115 (+ 343us) client_negotiation.cc:769] Sending connection context
0620 20:11:43.578348 (+ 233us) client_negotiation.cc:240] Negotiation successful
0620 20:11:43.578708 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":97,"threads_started":1}
I20260620 20:11:43.581211 21115 heartbeater.cc:347] Connected to a master server at 127.5.91.190:46243
I20260620 20:11:43.581481 21115 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:43.582058 21115 heartbeater.cc:511] Master 127.5.91.190:46243 requested a full tablet report, sending...
I20260620 20:11:43.583691 20921 ts_manager.cc:194] Registered new tserver with Master: 8d92933d8838494bb957c61d72325363 (127.5.91.129:38321)
I20260620 20:11:43.584867 20921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:49647
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
I20260620 20:11:43.593050 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46243
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:11:43.702896 21120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:43.703154 21120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:43.703215 21120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:43.707494 21120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:43.707572 21120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:43.707655 21120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:11:43.713150 21120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:43.714279 21120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:43.715272 21120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:43.724864 21128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:43.724946 21125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:43.724952 21126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:43.725214 21120 server_base.cc:1034] running on GCE node
I20260620 20:11:43.725638 21120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:43.726300 21120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:43.727496 21120 hybrid_clock.cc:648] HybridClock initialized: now 1781986303727442 us; error 64 us; skew 500 ppm
Jun 20 20:11:43 dist-test-slave-l8h4 krb5kdc[20878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986303, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:43.730899 21120 init.cc:377] Logged in from keytab as kudu/127.5.91.130@KRBTEST.COM (short username kudu)
I20260620 20:11:43.731805 21120 webserver.cc:460] Webserver started at http://127.5.91.130:35985/ using document root <none> and password file <none>
I20260620 20:11:43.732336 21120 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:43.732396 21120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:43.732611 21120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:11:43.734692 21120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "1da99bb317f4475a9bba1281755d974a"
format_stamp: "Formatted at 2026-06-20 20:11:43 on dist-test-slave-l8h4"
server_key: "981ea23d7c9452b34387a8936b699a7c"
server_key_iv: "c9f01cbad968e9920f491b8267a036d2"
server_key_version: "encryptionkey@0"
I20260620 20:11:43.735262 21120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "1da99bb317f4475a9bba1281755d974a"
format_stamp: "Formatted at 2026-06-20 20:11:43 on dist-test-slave-l8h4"
server_key: "981ea23d7c9452b34387a8936b699a7c"
server_key_iv: "c9f01cbad968e9920f491b8267a036d2"
server_key_version: "encryptionkey@0"
I20260620 20:11:43.738875 21120 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260620 20:11:43.741146 21135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:43.742373 21120 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:11:43.742465 21120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "1da99bb317f4475a9bba1281755d974a"
format_stamp: "Formatted at 2026-06-20 20:11:43 on dist-test-slave-l8h4"
server_key: "981ea23d7c9452b34387a8936b699a7c"
server_key_iv: "c9f01cbad968e9920f491b8267a036d2"
server_key_version: "encryptionkey@0"
I20260620 20:11:43.742597 21120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:43.764262 21120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:43.767171 21120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:43.767318 21120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:43.767812 21120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:11:43.768551 21120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:11:43.768596 21120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:43.768633 21120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:11:43.768653 21120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:43.778282 21120 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:44031
I20260620 20:11:43.778373 21249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:44031 every 8 connection(s)
I20260620 20:11:43.779356 21120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:11:43.780025 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 21120
I20260620 20:11:43.780143 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:11:43.780455 5486 external_mini_cluster.cc:1428] Setting key b234881756be789969ad82b94143b056
Jun 20 20:11:43 dist-test-slave-l8h4 krb5kdc[20878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986303, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.91.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.131@KRBTEST.COM" created.
I20260620 20:11:43.794862 20962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:43.781287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:56991 (local address 127.5.91.190:46243)
0620 20:11:43.781420 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:43.781424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:43.782435 (+ 1011us) server_negotiation.cc:407] Connection header received
0620 20:11:43.784011 (+ 1576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:43.784014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:43.784066 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:43.784135 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:43.785229 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:43.785970 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:43.787133 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:43.787509 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:43.787643 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:43.790623 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:43.790626 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:43.790629 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:43.790652 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:43.791852 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:43.792437 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:43.792440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:43.792441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:43.792531 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:43.793267 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:43.793272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:43.793273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:43.793577 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:43.793697 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:43.794473 (+ 776us) server_negotiation.cc:299] Negotiation successful
0620 20:11:43.794656 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 20:11:43.796090 21252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:43.781770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.130:56991)
0620 20:11:43.782290 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:43.782345 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:43.783794 (+ 1449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:43.784292 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:43.784299 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:43.784632 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:43.785095 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:43.785108 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:43.786100 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:43.786103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:43.787013 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:43.787023 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:43.787831 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:43.787836 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:43.788082 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:43.788807 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:43.788825 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:43.790457 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:43.792094 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:43.792100 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:43.792111 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:43.792351 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:43.792749 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:43.792754 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:43.792756 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:43.793083 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:43.793762 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:43.793771 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:43.794116 (+ 345us) client_negotiation.cc:769] Sending connection context
0620 20:11:43.794387 (+ 271us) client_negotiation.cc:240] Negotiation successful
0620 20:11:43.794833 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":122,"threads_started":1}
I20260620 20:11:43.797621 21250 heartbeater.cc:347] Connected to a master server at 127.5.91.190:46243
I20260620 20:11:43.797914 21250 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:43.798606 21250 heartbeater.cc:511] Master 127.5.91.190:46243 requested a full tablet report, sending...
I20260620 20:11:43.799799 20921 ts_manager.cc:194] Registered new tserver with Master: 1da99bb317f4475a9bba1281755d974a (127.5.91.130:44031)
I20260620 20:11:43.800551 20921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:56991
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
I20260620 20:11:43.809168 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46243
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:11:43.915889 21255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:11:43.916143 21255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:11:43.916184 21255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:11:43.920109 21255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:11:43.920166 21255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:11:43.920222 21255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:11:43.925545 21255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:11:43.926540 21255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:11:43.927448 21255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:11:43.936403 21263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:43.936404 21260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:11:43.936403 21261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:11:43.936651 21255 server_base.cc:1034] running on GCE node
I20260620 20:11:43.937110 21255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:11:43.937697 21255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:11:43.938872 21255 hybrid_clock.cc:648] HybridClock initialized: now 1781986303938847 us; error 37 us; skew 500 ppm
Jun 20 20:11:43 dist-test-slave-l8h4 krb5kdc[20878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986303, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:11:43.941505 21255 init.cc:377] Logged in from keytab as kudu/127.5.91.131@KRBTEST.COM (short username kudu)
I20260620 20:11:43.942335 21255 webserver.cc:460] Webserver started at http://127.5.91.131:38181/ using document root <none> and password file <none>
I20260620 20:11:43.942842 21255 fs_manager.cc:359] Metadata directory not provided
I20260620 20:11:43.942904 21255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:11:43.943079 21255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:11:43.945072 21255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "c298adebc87e4fc09a967bb4f5e6fe79"
format_stamp: "Formatted at 2026-06-20 20:11:43 on dist-test-slave-l8h4"
server_key: "eae95c15021f0077ba6eed0b9faad7e3"
server_key_iv: "3e2c399ba0395542b9f2bb8ee4d15dcd"
server_key_version: "encryptionkey@0"
I20260620 20:11:43.945580 21255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "c298adebc87e4fc09a967bb4f5e6fe79"
format_stamp: "Formatted at 2026-06-20 20:11:43 on dist-test-slave-l8h4"
server_key: "eae95c15021f0077ba6eed0b9faad7e3"
server_key_iv: "3e2c399ba0395542b9f2bb8ee4d15dcd"
server_key_version: "encryptionkey@0"
I20260620 20:11:43.949005 21255 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 20:11:43.951260 21270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:43.952365 21255 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 20:11:43.952471 21255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "c298adebc87e4fc09a967bb4f5e6fe79"
format_stamp: "Formatted at 2026-06-20 20:11:43 on dist-test-slave-l8h4"
server_key: "eae95c15021f0077ba6eed0b9faad7e3"
server_key_iv: "3e2c399ba0395542b9f2bb8ee4d15dcd"
server_key_version: "encryptionkey@0"
I20260620 20:11:43.952561 21255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:11:43.965893 21255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:11:43.968931 21255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:11:43.969151 21255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:11:43.969696 21255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:11:43.970503 21255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:11:43.970551 21255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:43.970602 21255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:11:43.970630 21255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:11:43.979619 21255 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:33369
I20260620 20:11:43.979636 21384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:33369 every 8 connection(s)
I20260620 20:11:43.980469 21255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:11:43.985168 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 21255
I20260620 20:11:43.985289 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:11:43.985574 5486 external_mini_cluster.cc:1428] Setting key c0c3763f28352a5d9044c721b580fdc9
Jun 20 20:11:43 dist-test-slave-l8h4 krb5kdc[20878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986303, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:11:43.993896 20962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:43.982268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:49771 (local address 127.5.91.190:46243)
0620 20:11:43.982399 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:43.982402 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:43.983255 (+ 853us) server_negotiation.cc:407] Connection header received
0620 20:11:43.984462 (+ 1207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:43.984465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:43.984521 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:43.984593 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:43.986004 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:43.986671 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:43.987941 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:43.988170 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:43.988255 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:43.990680 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:43.990686 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:43.990688 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:43.990710 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:43.991956 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:43.992449 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:43.992453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:43.992455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:43.992532 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:43.992893 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:43.992896 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:43.992897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:43.993109 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:43.993190 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:43.993666 (+ 476us) server_negotiation.cc:299] Negotiation successful
0620 20:11:43.993770 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 20:11:43.994724 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:43.982586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.131:49771)
0620 20:11:43.983105 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:43.983147 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:43.984268 (+ 1121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:43.984754 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:43.984760 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:43.985206 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:11:43.985852 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:43.985873 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:43.986820 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:43.986824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:43.987822 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:43.987833 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:43.988308 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:43.988310 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:43.988451 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:43.989008 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:11:43.989028 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:11:43.990510 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:11:43.992089 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:43.992098 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:43.992109 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:43.992352 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:43.992639 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:11:43.992641 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:11:43.992643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:11:43.992789 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:11:43.993207 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:11:43.993212 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:11:43.993434 (+ 222us) client_negotiation.cc:769] Sending connection context
0620 20:11:43.993597 (+ 163us) client_negotiation.cc:240] Negotiation successful
0620 20:11:43.993841 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":106,"threads_started":1}
I20260620 20:11:43.995877 21385 heartbeater.cc:347] Connected to a master server at 127.5.91.190:46243
I20260620 20:11:43.996130 21385 heartbeater.cc:464] Registering TS with master...
I20260620 20:11:43.996600 21385 heartbeater.cc:511] Master 127.5.91.190:46243 requested a full tablet report, sending...
I20260620 20:11:43.997520 20921 ts_manager.cc:194] Registered new tserver with Master: c298adebc87e4fc09a967bb4f5e6fe79 (127.5.91.131:33369)
I20260620 20:11:43.998124 20921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:49771
I20260620 20:11:43.999471 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 20:11:44.009126 20962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:44.001532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36442 (local address 127.5.91.190:46243)
0620 20:11:44.001773 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:44.001777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:44.001910 (+ 133us) server_negotiation.cc:407] Connection header received
0620 20:11:44.002047 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:44.002049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:44.002092 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:44.002175 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:11:44.002828 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.003406 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.004399 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.004663 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.004750 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:44.006076 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:11:44.006080 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:11:44.006082 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:11:44.006106 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:11:44.007626 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:44.008004 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:44.008007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:44.008009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:44.008084 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:11:44.008309 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:11:44.008312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:11:44.008314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:11:44.008535 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:11:44.008649 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:44.008859 (+ 210us) server_negotiation.cc:299] Negotiation successful
0620 20:11:44.008966 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260620 20:11:44.012565 20921 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:11:44.015585 20921 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:11:44.029335 21397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:44.023702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:33369 (local address 127.0.0.1:51824)
0620 20:11:44.024535 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:44.024582 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:44.024711 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:44.025220 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:44.025226 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:44.025246 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:11:44.025375 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:44.025382 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.026548 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.026551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:44.027766 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:44.027776 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.028812 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.028815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:44.028930 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:44.028971 (+ 41us) client_negotiation.cc:769] Sending connection context
0620 20:11:44.029068 (+ 97us) client_negotiation.cc:240] Negotiation successful
0620 20:11:44.029178 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"mutex_wait_us":24,"thread_start_us":66,"threads_started":1}
I20260620 20:11:44.029515 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:44.023795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:38321 (local address 127.0.0.1:49080)
0620 20:11:44.024582 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:44.024606 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:44.024711 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:44.025572 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:44.025574 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:44.025588 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:11:44.025674 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:44.025678 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.027004 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.027007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:44.028104 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:44.028113 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.029099 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.029102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:44.029188 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:44.029209 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 20:11:44.029282 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 20:11:44.029348 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"mutex_wait_us":1,"thread_start_us":87,"threads_started":1}
I20260620 20:11:44.029767 21401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:44.024516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49080 (local address 127.5.91.129:38321)
0620 20:11:44.024994 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:44.025008 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:44.025027 (+ 19us) server_negotiation.cc:407] Connection header received
0620 20:11:44.025095 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:44.025099 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:44.025265 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:44.025417 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:11:44.025781 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.026864 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.028228 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.028994 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.029087 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:44.029210 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:44.029311 (+ 101us) server_negotiation.cc:299] Negotiation successful
0620 20:11:44.029403 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":93,"threads_started":1}
I20260620 20:11:44.029955 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:44.023821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51824 (local address 127.5.91.131:33369)
0620 20:11:44.024167 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:44.024179 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:44.024722 (+ 543us) server_negotiation.cc:407] Connection header received
0620 20:11:44.024846 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:44.024852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:44.025078 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:44.025230 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:11:44.025479 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.026432 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.027898 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.028706 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.028816 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:44.028959 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:44.029704 (+ 745us) server_negotiation.cc:299] Negotiation successful
0620 20:11:44.029802 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":160,"threads_started":1}
I20260620 20:11:44.030244 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:44.024299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:44031 (local address 127.0.0.1:55430)
0620 20:11:44.025848 (+ 1549us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:44.025861 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:44.025942 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:44.026346 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:44.026349 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:44.026367 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:11:44.026452 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:44.026458 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.027679 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.027682 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:44.028611 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:44.028620 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.029865 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.029868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:44.029952 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:44.029974 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 20:11:44.030039 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 20:11:44.030105 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1491,"thread_start_us":87,"threads_started":1}
I20260620 20:11:44.030596 21400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:44.024447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55430 (local address 127.5.91.130:44031)
0620 20:11:44.024994 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:44.025005 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:44.025945 (+ 940us) server_negotiation.cc:407] Connection header received
0620 20:11:44.026028 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:44.026033 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:44.026217 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:44.026344 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:11:44.026576 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.027446 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.028728 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.029596 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.029688 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:44.029806 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:44.030406 (+ 600us) server_negotiation.cc:299] Negotiation successful
0620 20:11:44.030486 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":95,"threads_started":1}
I20260620 20:11:44.032595 21049 tablet_service.cc:1467] Processing CreateTablet for tablet 9dfebc6fb4ed4e80911e8bc02fcf7bda (DEFAULT_TABLE table=test-table [id=49ea806060684196966c7a7fb889669d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:11:44.032855 21183 tablet_service.cc:1467] Processing CreateTablet for tablet 9dfebc6fb4ed4e80911e8bc02fcf7bda (DEFAULT_TABLE table=test-table [id=49ea806060684196966c7a7fb889669d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:11:44.033576 21049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dfebc6fb4ed4e80911e8bc02fcf7bda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:11:44.033697 21183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dfebc6fb4ed4e80911e8bc02fcf7bda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:11:44.036280 21318 tablet_service.cc:1467] Processing CreateTablet for tablet 9dfebc6fb4ed4e80911e8bc02fcf7bda (DEFAULT_TABLE table=test-table [id=49ea806060684196966c7a7fb889669d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:11:44.037319 21318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dfebc6fb4ed4e80911e8bc02fcf7bda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:11:44.039383 21402 tablet_bootstrap.cc:492] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 1da99bb317f4475a9bba1281755d974a: Bootstrap starting.
I20260620 20:11:44.041671 21403 tablet_bootstrap.cc:492] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363: Bootstrap starting.
I20260620 20:11:44.041950 21402 tablet_bootstrap.cc:654] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 1da99bb317f4475a9bba1281755d974a: Neither blocks nor log segments found. Creating new log.
I20260620 20:11:44.042817 21402 log.cc:824] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 1da99bb317f4475a9bba1281755d974a: Log is configured to *not* fsync() on all Append() calls
I20260620 20:11:44.043712 21403 tablet_bootstrap.cc:654] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363: Neither blocks nor log segments found. Creating new log.
I20260620 20:11:44.044806 21403 log.cc:824] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363: Log is configured to *not* fsync() on all Append() calls
I20260620 20:11:44.044848 21402 tablet_bootstrap.cc:492] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 1da99bb317f4475a9bba1281755d974a: No bootstrap required, opened a new log
I20260620 20:11:44.045035 21402 ts_tablet_manager.cc:1397] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 1da99bb317f4475a9bba1281755d974a: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260620 20:11:44.046424 21405 tablet_bootstrap.cc:492] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P c298adebc87e4fc09a967bb4f5e6fe79: Bootstrap starting.
I20260620 20:11:44.046748 21403 tablet_bootstrap.cc:492] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363: No bootstrap required, opened a new log
I20260620 20:11:44.046921 21403 ts_tablet_manager.cc:1397] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260620 20:11:44.048110 21405 tablet_bootstrap.cc:654] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P c298adebc87e4fc09a967bb4f5e6fe79: Neither blocks nor log segments found. Creating new log.
I20260620 20:11:44.048976 21405 log.cc:824] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P c298adebc87e4fc09a967bb4f5e6fe79: Log is configured to *not* fsync() on all Append() calls
I20260620 20:11:44.048836 21402 raft_consensus.cc:348] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 1da99bb317f4475a9bba1281755d974a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } }
I20260620 20:11:44.049098 21402 raft_consensus.cc:374] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 1da99bb317f4475a9bba1281755d974a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:11:44.049180 21402 raft_consensus.cc:729] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 1da99bb317f4475a9bba1281755d974a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1da99bb317f4475a9bba1281755d974a, State: Initialized, Role: FOLLOWER
I20260620 20:11:44.049615 21402 consensus_queue.cc:260] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 1da99bb317f4475a9bba1281755d974a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } }
I20260620 20:11:44.050257 21250 heartbeater.cc:503] Master 127.5.91.190:46243 was elected leader, sending a full tablet report...
I20260620 20:11:44.050629 21402 ts_tablet_manager.cc:1428] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 1da99bb317f4475a9bba1281755d974a: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260620 20:11:44.050504 21403 raft_consensus.cc:348] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } }
I20260620 20:11:44.050729 21403 raft_consensus.cc:374] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:11:44.050734 21405 tablet_bootstrap.cc:492] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P c298adebc87e4fc09a967bb4f5e6fe79: No bootstrap required, opened a new log
I20260620 20:11:44.050798 21403 raft_consensus.cc:729] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d92933d8838494bb957c61d72325363, State: Initialized, Role: FOLLOWER
I20260620 20:11:44.050908 21405 ts_tablet_manager.cc:1397] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P c298adebc87e4fc09a967bb4f5e6fe79: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 20:11:44.051246 21403 consensus_queue.cc:260] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } }
I20260620 20:11:44.052171 21403 ts_tablet_manager.cc:1428] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260620 20:11:44.052171 21115 heartbeater.cc:503] Master 127.5.91.190:46243 was elected leader, sending a full tablet report...
I20260620 20:11:44.054368 21405 raft_consensus.cc:348] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P c298adebc87e4fc09a967bb4f5e6fe79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } }
I20260620 20:11:44.054667 21405 raft_consensus.cc:374] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P c298adebc87e4fc09a967bb4f5e6fe79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:11:44.054729 21405 raft_consensus.cc:729] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P c298adebc87e4fc09a967bb4f5e6fe79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c298adebc87e4fc09a967bb4f5e6fe79, State: Initialized, Role: FOLLOWER
I20260620 20:11:44.055150 21405 consensus_queue.cc:260] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P c298adebc87e4fc09a967bb4f5e6fe79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } }
I20260620 20:11:44.055727 21385 heartbeater.cc:503] Master 127.5.91.190:46243 was elected leader, sending a full tablet report...
I20260620 20:11:44.055981 21405 ts_tablet_manager.cc:1428] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P c298adebc87e4fc09a967bb4f5e6fe79: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
W20260620 20:11:44.062307 21116 tablet.cc:2369] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:11:44.180083 21409 raft_consensus.cc:482] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:11:44.180331 21409 raft_consensus.cc:504] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } }
I20260620 20:11:44.181478 21409 leader_election.cc:288] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c298adebc87e4fc09a967bb4f5e6fe79 (127.5.91.131:33369), 1da99bb317f4475a9bba1281755d974a (127.5.91.130:44031)
I20260620 20:11:44.185050 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:44.181827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:42611 (local address 127.5.91.131:33369)
0620 20:11:44.182080 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:44.182084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:44.182099 (+ 15us) server_negotiation.cc:407] Connection header received
0620 20:11:44.182251 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:44.182253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:44.182295 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:44.182352 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:11:44.182623 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.183122 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.184142 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.184535 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.184635 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:44.184688 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:44.184845 (+ 157us) server_negotiation.cc:299] Negotiation successful
0620 20:11:44.184909 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260620 20:11:44.185070 21400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:44.181851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:57825 (local address 127.5.91.130:44031)
0620 20:11:44.181992 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:44.181996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:44.182215 (+ 219us) server_negotiation.cc:407] Connection header received
0620 20:11:44.182344 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:44.182347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:44.182393 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:44.182454 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:11:44.182700 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.183211 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.184173 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.184556 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.184655 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:44.184704 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:44.184873 (+ 169us) server_negotiation.cc:299] Negotiation successful
0620 20:11:44.184941 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 20:11:44.185086 21412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:44.181705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:33369 (local address 127.5.91.129:42611)
0620 20:11:44.182008 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:44.182023 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:44.182125 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:44.182378 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:44.182380 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:44.182397 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:11:44.182522 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:44.182528 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.183223 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.183226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:44.184026 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:44.184035 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.184641 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.184644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:44.184736 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:44.184758 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 20:11:44.184820 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 20:11:44.184909 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":3,"thread_start_us":82,"threads_started":1}
I20260620 20:11:44.185091 21411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:44.181711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:44031 (local address 127.5.91.129:57825)
0620 20:11:44.182126 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:44.182156 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0620 20:11:44.182250 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:11:44.182470 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:11:44.182472 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:11:44.182496 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:11:44.182607 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:44.182611 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.183312 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.183315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:44.184062 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:11:44.184070 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.184658 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.184662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:11:44.184764 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:44.184785 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 20:11:44.184846 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 20:11:44.184942 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":31,"thread_start_us":103,"threads_started":1}
I20260620 20:11:44.185706 21203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dfebc6fb4ed4e80911e8bc02fcf7bda" candidate_uuid: "8d92933d8838494bb957c61d72325363" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1da99bb317f4475a9bba1281755d974a" is_pre_election: true
I20260620 20:11:44.185705 21339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dfebc6fb4ed4e80911e8bc02fcf7bda" candidate_uuid: "8d92933d8838494bb957c61d72325363" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" is_pre_election: true
I20260620 20:11:44.186002 21339 raft_consensus.cc:2454] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P c298adebc87e4fc09a967bb4f5e6fe79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d92933d8838494bb957c61d72325363 in term 0.
I20260620 20:11:44.186000 21203 raft_consensus.cc:2454] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 1da99bb317f4475a9bba1281755d974a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d92933d8838494bb957c61d72325363 in term 0.
I20260620 20:11:44.186489 21004 leader_election.cc:302] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d92933d8838494bb957c61d72325363, c298adebc87e4fc09a967bb4f5e6fe79; no voters:
I20260620 20:11:44.186764 21409 raft_consensus.cc:2781] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:11:44.186837 21409 raft_consensus.cc:482] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:11:44.186868 21409 raft_consensus.cc:3037] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:11:44.187885 21409 raft_consensus.cc:504] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } }
I20260620 20:11:44.188212 21409 leader_election.cc:288] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 1 election: Requested vote from peers c298adebc87e4fc09a967bb4f5e6fe79 (127.5.91.131:33369), 1da99bb317f4475a9bba1281755d974a (127.5.91.130:44031)
I20260620 20:11:44.188544 21203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dfebc6fb4ed4e80911e8bc02fcf7bda" candidate_uuid: "8d92933d8838494bb957c61d72325363" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1da99bb317f4475a9bba1281755d974a"
I20260620 20:11:44.188562 21339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dfebc6fb4ed4e80911e8bc02fcf7bda" candidate_uuid: "8d92933d8838494bb957c61d72325363" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c298adebc87e4fc09a967bb4f5e6fe79"
I20260620 20:11:44.188647 21203 raft_consensus.cc:3037] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 1da99bb317f4475a9bba1281755d974a [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:11:44.188688 21339 raft_consensus.cc:3037] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P c298adebc87e4fc09a967bb4f5e6fe79 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:11:44.189832 21339 raft_consensus.cc:2454] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P c298adebc87e4fc09a967bb4f5e6fe79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d92933d8838494bb957c61d72325363 in term 1.
I20260620 20:11:44.189857 21203 raft_consensus.cc:2454] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 1da99bb317f4475a9bba1281755d974a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d92933d8838494bb957c61d72325363 in term 1.
I20260620 20:11:44.190109 21004 leader_election.cc:302] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d92933d8838494bb957c61d72325363, c298adebc87e4fc09a967bb4f5e6fe79; no voters:
I20260620 20:11:44.190295 21409 raft_consensus.cc:2781] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:11:44.190523 21409 raft_consensus.cc:686] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [term 1 LEADER]: Becoming Leader. State: Replica: 8d92933d8838494bb957c61d72325363, State: Running, Role: LEADER
I20260620 20:11:44.190860 21409 consensus_queue.cc:237] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } }
I20260620 20:11:44.193801 20921 catalog_manager.cc:5495] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d92933d8838494bb957c61d72325363 (127.5.91.129). New cstate: current_term: 1 leader_uuid: "8d92933d8838494bb957c61d72325363" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } health_report { overall_health: UNKNOWN } } }
I20260620 20:11:44.226114 21401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:44.222384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49088 (local address 127.5.91.129:38321)
0620 20:11:44.222507 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0620 20:11:44.222511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:11:44.222524 (+ 13us) server_negotiation.cc:407] Connection header received
0620 20:11:44.222653 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:11:44.222655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:11:44.222712 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:11:44.222808 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:11:44.223078 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.223700 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.224584 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:11:44.224792 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:11:44.224888 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:11:44.225192 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:11:44.225627 (+ 435us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:11:44.225734 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 20:11:44.225812 (+ 78us) server_negotiation.cc:299] Negotiation successful
0620 20:11:44.225874 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260620 20:11:44.231374 21386 tablet.cc:2369] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P c298adebc87e4fc09a967bb4f5e6fe79: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:11:44.232062 21203 raft_consensus.cc:1261] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 1da99bb317f4475a9bba1281755d974a [term 1 FOLLOWER]: Refusing update from remote peer 8d92933d8838494bb957c61d72325363: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:11:44.232098 21339 raft_consensus.cc:1261] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P c298adebc87e4fc09a967bb4f5e6fe79 [term 1 FOLLOWER]: Refusing update from remote peer 8d92933d8838494bb957c61d72325363: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:11:44.232757 21413 consensus_queue.cc:1035] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:11:44.232925 21409 consensus_queue.cc:1035] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:11:44.239555 21416 mvcc.cc:205] Tried to move back new op lower bound from 7299015902127349760 to 7299015901968310272. Current Snapshot: MvccSnapshot[applied={T|T < 7299015902127349760}]
I20260620 20:11:44.247932 20921 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:11:44.248128 20921 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:11:44.250699 20921 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90: Sending DeleteTablet for 3 replicas of tablet 9dfebc6fb4ed4e80911e8bc02fcf7bda
I20260620 20:11:44.251468 21318 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dfebc6fb4ed4e80911e8bc02fcf7bda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:11:44 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:51824
I20260620 20:11:44.251473 21183 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dfebc6fb4ed4e80911e8bc02fcf7bda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:11:44 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:55430
I20260620 20:11:44.251470 21049 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dfebc6fb4ed4e80911e8bc02fcf7bda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:11:44 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:49080
I20260620 20:11:44.252000 21426 tablet_replica.cc:330] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P c298adebc87e4fc09a967bb4f5e6fe79: stopping tablet replica
I20260620 20:11:44.252238 21427 tablet_replica.cc:330] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 1da99bb317f4475a9bba1281755d974a: stopping tablet replica
I20260620 20:11:44.252349 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 20985
I20260620 20:11:44.252581 21427 raft_consensus.cc:2229] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 1da99bb317f4475a9bba1281755d974a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:11:44.252914 21427 raft_consensus.cc:2258] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 1da99bb317f4475a9bba1281755d974a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:11:44.253134 21426 raft_consensus.cc:2229] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P c298adebc87e4fc09a967bb4f5e6fe79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:11:44.253441 21426 raft_consensus.cc:2258] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P c298adebc87e4fc09a967bb4f5e6fe79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:11:44.254518 21427 ts_tablet_manager.cc:1905] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 1da99bb317f4475a9bba1281755d974a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:11:44.254846 21426 ts_tablet_manager.cc:1905] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P c298adebc87e4fc09a967bb4f5e6fe79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:11:44.257190 21426 ts_tablet_manager.cc:1918] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P c298adebc87e4fc09a967bb4f5e6fe79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:11:44.257277 21426 log.cc:1196] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P c298adebc87e4fc09a967bb4f5e6fe79: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/9dfebc6fb4ed4e80911e8bc02fcf7bda
I20260620 20:11:44.257381 21427 ts_tablet_manager.cc:1918] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 1da99bb317f4475a9bba1281755d974a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:11:44.257472 21427 log.cc:1196] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 1da99bb317f4475a9bba1281755d974a: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/wal/wals/9dfebc6fb4ed4e80911e8bc02fcf7bda
I20260620 20:11:44.257615 21426 ts_tablet_manager.cc:1939] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P c298adebc87e4fc09a967bb4f5e6fe79: Deleting consensus metadata
I20260620 20:11:44.257807 21427 ts_tablet_manager.cc:1939] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 1da99bb317f4475a9bba1281755d974a: Deleting consensus metadata
I20260620 20:11:44.258818 20910 catalog_manager.cc:4841] TS c298adebc87e4fc09a967bb4f5e6fe79 (127.5.91.131:33369): tablet 9dfebc6fb4ed4e80911e8bc02fcf7bda (table test-table [id=49ea806060684196966c7a7fb889669d]) successfully deleted
I20260620 20:11:44.259155 20907 catalog_manager.cc:4841] TS 1da99bb317f4475a9bba1281755d974a (127.5.91.130:44031): tablet 9dfebc6fb4ed4e80911e8bc02fcf7bda (table test-table [id=49ea806060684196966c7a7fb889669d]) successfully deleted
W20260620 20:11:44.260013 20909 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.91.129:38321 (error 108)
I20260620 20:11:44.260262 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 21120
I20260620 20:11:44.261585 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:11:44.260935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:38321 (local address 127.0.0.1:49104)
0620 20:11:44.261456 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0620 20:11:44.261514 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.129:38321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446}
W20260620 20:11:44.261732 20909 catalog_manager.cc:4584] TS 8d92933d8838494bb957c61d72325363 (127.5.91.129:38321): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9dfebc6fb4ed4e80911e8bc02fcf7bda: Network error: Client connection negotiation failed: client connection to 127.5.91.129:38321: connect: Connection refused (error 111)
I20260620 20:11:44.265679 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 21255
I20260620 20:11:44.270534 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 20891
I20260620 20:12:04.276427 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:46243
--webserver_interface=127.5.91.190
--webserver_port=37723
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:46243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:12:04.385944 21430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:04.386222 21430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:04.386265 21430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:04.390239 21430 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 20:12:04.390295 21430 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 20:12:04.390313 21430 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 20:12:04.390331 21430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:12:04.390352 21430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:04.390389 21430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:04.390425 21430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:12:04.390450 21430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:12:04.395681 21430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:46243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:46243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=37723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:04.396677 21430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:04.397652 21430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:04.410804 21437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:04.410804 21436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:04.411163 21430 server_base.cc:1034] running on GCE node
W20260620 20:12:04.411206 21439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:04.411847 21430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:04.413054 21430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:04.414234 21430 hybrid_clock.cc:648] HybridClock initialized: now 1781986324414209 us; error 41 us; skew 500 ppm
Jun 20 20:12:04 dist-test-slave-l8h4 krb5kdc[20878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986324, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:04.417025 21430 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:12:04.417977 21430 webserver.cc:460] Webserver started at http://127.5.91.190:37723/ using document root <none> and password file <none>
I20260620 20:12:04.418596 21430 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:04.418680 21430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:04.422545 21430 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 20:12:04.424715 21446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:04.426249 21430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:12:04.426386 21430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "f6478905ca1f4e50b944e5274c384f90"
format_stamp: "Formatted at 2026-06-20 20:11:43 on dist-test-slave-l8h4"
server_key: "e71e575c547b495726c25ec2bd78000d"
server_key_iv: "4bdbda48ef9ac917c0e28347a60d7587"
server_key_version: "encryptionkey@0"
I20260620 20:12:04.426988 21430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:04.452266 21430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:04.455549 21430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:04.455739 21430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:04.462922 21430 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:46243
I20260620 20:12:04.463294 21499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:46243 every 8 connection(s)
I20260620 20:12:04.463986 21430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:12:04.468240 21500 sys_catalog.cc:263] Verifying existing consensus state
I20260620 20:12:04.469800 21500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90: Bootstrap starting.
I20260620 20:12:04.473587 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 21430
I20260620 20:12:04.474037 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:38321
--local_ip_for_outbound_sockets=127.5.91.129
--tserver_master_addrs=127.5.91.190:46243
--webserver_port=36389
--webserver_interface=127.5.91.129
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 20:12:04.487461 21500 log.cc:824] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:04.496263 21500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 20:12:04.496603 21500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90: Bootstrap complete.
I20260620 20:12:04.500509 21500 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6478905ca1f4e50b944e5274c384f90" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46243 } }
I20260620 20:12:04.500984 21500 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6478905ca1f4e50b944e5274c384f90, State: Initialized, Role: FOLLOWER
I20260620 20:12:04.501435 21500 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6478905ca1f4e50b944e5274c384f90" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46243 } }
I20260620 20:12:04.501554 21500 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:12:04.501628 21500 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:12:04.501713 21500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 1 FOLLOWER]: Advancing to term 2
I20260620 20:12:04.503681 21500 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6478905ca1f4e50b944e5274c384f90" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46243 } }
I20260620 20:12:04.504045 21500 leader_election.cc:302] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6478905ca1f4e50b944e5274c384f90; no voters:
I20260620 20:12:04.504371 21500 leader_election.cc:288] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 20:12:04.504475 21504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 20:12:04.504637 21504 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 2 LEADER]: Becoming Leader. State: Replica: f6478905ca1f4e50b944e5274c384f90, State: Running, Role: LEADER
I20260620 20:12:04.504980 21504 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6478905ca1f4e50b944e5274c384f90" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46243 } }
I20260620 20:12:04.505590 21500 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:12:04.505980 21504 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6478905ca1f4e50b944e5274c384f90. Latest consensus state: current_term: 2 leader_uuid: "f6478905ca1f4e50b944e5274c384f90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6478905ca1f4e50b944e5274c384f90" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46243 } } }
I20260620 20:12:04.506318 21504 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:04.507119 21505 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f6478905ca1f4e50b944e5274c384f90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6478905ca1f4e50b944e5274c384f90" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46243 } } }
I20260620 20:12:04.507414 21505 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:04.510258 21519 catalog_manager.cc:1259] Loaded cluster ID: 6f3b3f9eeb994d8384ed0ab5c21539d8
I20260620 20:12:04.510313 21519 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90: loading cluster ID for follower catalog manager: success
I20260620 20:12:04.512439 21519 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90: acquiring CA information for follower catalog manager: success
I20260620 20:12:04.513602 21519 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 20:12:04.513701 21513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:12:04.515583 21513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:12:04.515997 21513 catalog_manager.cc:1259] Loaded cluster ID: 6f3b3f9eeb994d8384ed0ab5c21539d8
I20260620 20:12:04.516050 21513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:12:04.516551 21513 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:12:04.516925 21513 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90: Loaded TSK: 0
I20260620 20:12:04.525259 21513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90: Generated new TSK 1
I20260620 20:12:04.525683 21513 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 20:12:04.599187 21502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:04.599431 21502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:04.599472 21502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:04.603560 21502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:04.603621 21502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:04.603683 21502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:12:04.609031 21502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:38321
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=36389
--tserver_master_addrs=127.5.91.190:46243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:04.609936 21502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:04.610877 21502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 20:12:04.619956 21502 server_base.cc:1034] running on GCE node
W20260620 20:12:04.619912 21527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:04.619918 21528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:04.620272 21530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:04.620700 21502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:04.621313 21502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:04.622483 21502 hybrid_clock.cc:648] HybridClock initialized: now 1781986324622450 us; error 46 us; skew 500 ppm
Jun 20 20:12:04 dist-test-slave-l8h4 krb5kdc[20878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986324, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:04.625228 21502 init.cc:377] Logged in from keytab as kudu/127.5.91.129@KRBTEST.COM (short username kudu)
I20260620 20:12:04.626080 21502 webserver.cc:460] Webserver started at http://127.5.91.129:36389/ using document root <none> and password file <none>
I20260620 20:12:04.626621 21502 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:04.626690 21502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:04.630693 21502 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 20:12:04.632740 21537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:04.633919 21502 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 20:12:04.634013 21502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "8d92933d8838494bb957c61d72325363"
format_stamp: "Formatted at 2026-06-20 20:11:43 on dist-test-slave-l8h4"
server_key: "fe303842c202b2d9889ca90b71a35879"
server_key_iv: "37e16f5aaf9372e3e7c3830a3901a78e"
server_key_version: "encryptionkey@0"
I20260620 20:12:04.634573 21502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:04.662099 21502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:04.665517 21502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:04.665709 21502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:04.666388 21502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:04.667488 21544 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 20:12:04.671449 21502 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 20:12:04.671518 21502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260620 20:12:04.671577 21502 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 20:12:04.673301 21502 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 20:12:04.673349 21502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260620 20:12:04.673619 21544 tablet_bootstrap.cc:492] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363: Bootstrap starting.
I20260620 20:12:04.683729 21502 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:38321
I20260620 20:12:04.683792 21652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:38321 every 8 connection(s)
I20260620 20:12:04.684675 21502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:12:04.690928 21544 log.cc:824] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:04.691370 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 21502
I20260620 20:12:04.691828 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:44031
--local_ip_for_outbound_sockets=127.5.91.130
--tserver_master_addrs=127.5.91.190:46243
--webserver_port=35985
--webserver_interface=127.5.91.130
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 20:12:04 dist-test-slave-l8h4 krb5kdc[20878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986324, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:12:04.699115 21544 tablet_bootstrap.cc:492] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 20:12:04.699522 21544 tablet_bootstrap.cc:492] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363: Bootstrap complete.
I20260620 20:12:04.700583 21544 ts_tablet_manager.cc:1397] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363: Time spent bootstrapping tablet: real 0.027s user 0.021s sys 0.004s
I20260620 20:12:04.703445 21656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:04.686808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.129:60505)
0620 20:12:04.687295 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:04.687334 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:04.688569 (+ 1235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:04.689805 (+ 1236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:04.689814 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:04.690275 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:04.690761 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:04.690777 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:04.692515 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:04.692519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:04.693832 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:04.693843 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:04.694653 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:04.694657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:04.694833 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:04.695452 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:12:04.695471 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:04.697305 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:04.699938 (+ 2633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:04.699945 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:04.699960 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:04.700327 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:04.700814 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:04.700817 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:04.700818 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:04.700977 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:04.701702 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:04.701709 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:04.701975 (+ 266us) client_negotiation.cc:769] Sending connection context
0620 20:12:04.702192 (+ 217us) client_negotiation.cc:240] Negotiation successful
0620 20:12:04.702522 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":137,"threads_started":1}
I20260620 20:12:04.703478 21655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:04.686746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:60505 (local address 127.5.91.190:46243)
0620 20:12:04.687127 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:04.687147 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:04.687463 (+ 316us) server_negotiation.cc:407] Connection header received
0620 20:12:04.688996 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:04.689006 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:04.689288 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:04.689704 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:04.690931 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:04.692351 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:04.693968 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:04.694507 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:04.694641 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:04.697528 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:04.697535 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:04.697550 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:04.697647 (+ 97us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:04.699790 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:04.700499 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:04.700506 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:04.700512 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:04.700693 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:04.701083 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:04.701086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:04.701088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:04.701576 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:04.701724 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:04.702337 (+ 613us) server_negotiation.cc:299] Negotiation successful
0620 20:12:04.702663 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":102,"threads_started":1}
I20260620 20:12:04.703387 21544 raft_consensus.cc:348] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } }
I20260620 20:12:04.704032 21544 raft_consensus.cc:729] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d92933d8838494bb957c61d72325363, State: Initialized, Role: FOLLOWER
I20260620 20:12:04.704515 21544 consensus_queue.cc:260] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } }
I20260620 20:12:04.705490 21544 ts_tablet_manager.cc:1428] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260620 20:12:04.706614 21653 heartbeater.cc:347] Connected to a master server at 127.5.91.190:46243
I20260620 20:12:04.706861 21653 heartbeater.cc:464] Registering TS with master...
I20260620 20:12:04.707422 21653 heartbeater.cc:511] Master 127.5.91.190:46243 requested a full tablet report, sending...
I20260620 20:12:04.709946 21464 ts_manager.cc:194] Registered new tserver with Master: 8d92933d8838494bb957c61d72325363 (127.5.91.129:38321)
I20260620 20:12:04.713932 21464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:60505
I20260620 20:12:04.716872 21653 heartbeater.cc:503] Master 127.5.91.190:46243 was elected leader, sending a full tablet report...
Jun 20 20:12:04 dist-test-slave-l8h4 krb5kdc[20878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986324, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for kudu/127.5.91.129@KRBTEST.COM
I20260620 20:12:04.725677 21660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:04.713351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:38321 (local address 127.0.0.1:50314)
0620 20:12:04.714126 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:04.714193 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:04.714333 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:04.714851 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:04.714857 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:04.715224 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:04.715328 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:04.715335 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:04.716386 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:04.716390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:04.717036 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:04.717049 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:04.717768 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:04.717772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:04.717857 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:04.718561 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:12:04.718577 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:04.720395 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:04.723320 (+ 2925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:04.723350 (+ 30us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:04.723357 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:04.723701 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:04.724083 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:04.724085 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:04.724087 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:04.724184 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:04.724903 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:04.724911 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:04.725061 (+ 150us) client_negotiation.cc:769] Sending connection context
0620 20:12:04.725203 (+ 142us) client_negotiation.cc:240] Negotiation successful
0620 20:12:04.725481 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":102,"threads_started":1}
I20260620 20:12:04.725677 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:04.713456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50314 (local address 127.5.91.129:38321)
0620 20:12:04.713875 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:04.713887 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:04.714291 (+ 404us) server_negotiation.cc:407] Connection header received
0620 20:12:04.714473 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:04.714478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:04.714700 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:04.714830 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:04.715452 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:04.716264 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:04.717167 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:04.717524 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:04.717635 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:04.720551 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:04.720560 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:04.720570 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:04.720608 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:04.723128 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:04.723861 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:04.723867 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:04.723869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:04.723975 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:04.724414 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:04.724419 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:04.724421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:04.724752 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:04.724897 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:04.725310 (+ 413us) server_negotiation.cc:299] Negotiation successful
0620 20:12:04.725492 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":124,"threads_started":1}
I20260620 20:12:04.727145 21586 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dfebc6fb4ed4e80911e8bc02fcf7bda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:11:44 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:50314
I20260620 20:12:04.728519 21662 tablet_replica.cc:330] stopping tablet replica
I20260620 20:12:04.728823 21662 raft_consensus.cc:2229] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:12:04.729164 21662 raft_consensus.cc:2258] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:12:04.730197 21662 ts_tablet_manager.cc:1905] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:12:04.733706 21662 ts_tablet_manager.cc:1918] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:12:04.733793 21662 log.cc:1196] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/wal/wals/9dfebc6fb4ed4e80911e8bc02fcf7bda
I20260620 20:12:04.734462 21662 ts_tablet_manager.cc:1939] T 9dfebc6fb4ed4e80911e8bc02fcf7bda P 8d92933d8838494bb957c61d72325363: Deleting consensus metadata
I20260620 20:12:04.736600 21449 catalog_manager.cc:4841] TS 8d92933d8838494bb957c61d72325363 (127.5.91.129:38321): tablet 9dfebc6fb4ed4e80911e8bc02fcf7bda (table test-table [id=49ea806060684196966c7a7fb889669d]) successfully deleted
W20260620 20:12:04.810650 21658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:04.810894 21658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:04.810935 21658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:04.814860 21658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:04.814911 21658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:04.814975 21658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:12:04.820351 21658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:44031
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=35985
--tserver_master_addrs=127.5.91.190:46243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:04.821362 21658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:04.822347 21658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:04.831429 21668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:04.831437 21670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:04.831598 21658 server_base.cc:1034] running on GCE node
W20260620 20:12:04.831501 21667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:04.831979 21658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:04.832568 21658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:04.833742 21658 hybrid_clock.cc:648] HybridClock initialized: now 1781986324833719 us; error 35 us; skew 500 ppm
Jun 20 20:12:04 dist-test-slave-l8h4 krb5kdc[20878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986324, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:04.836308 21658 init.cc:377] Logged in from keytab as kudu/127.5.91.130@KRBTEST.COM (short username kudu)
I20260620 20:12:04.837183 21658 webserver.cc:460] Webserver started at http://127.5.91.130:35985/ using document root <none> and password file <none>
I20260620 20:12:04.837713 21658 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:04.837790 21658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:04.841624 21658 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260620 20:12:04.843879 21677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:04.845106 21658 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 20:12:04.845225 21658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "1da99bb317f4475a9bba1281755d974a"
format_stamp: "Formatted at 2026-06-20 20:11:43 on dist-test-slave-l8h4"
server_key: "981ea23d7c9452b34387a8936b699a7c"
server_key_iv: "c9f01cbad968e9920f491b8267a036d2"
server_key_version: "encryptionkey@0"
I20260620 20:12:04.845745 21658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:04.870695 21658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:04.874346 21658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:04.874540 21658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:04.875131 21658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:04.876085 21658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:12:04.876148 21658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:04.876205 21658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:12:04.876231 21658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:04.885196 21658 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:44031
I20260620 20:12:04.885211 21791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:44031 every 8 connection(s)
I20260620 20:12:04.886193 21658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:12:04.889426 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 21658
I20260620 20:12:04.889813 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:33369
--local_ip_for_outbound_sockets=127.5.91.131
--tserver_master_addrs=127.5.91.190:46243
--webserver_port=38181
--webserver_interface=127.5.91.131
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 20:12:04 dist-test-slave-l8h4 krb5kdc[20878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986324, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:12:04.899852 21655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:04.888061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:49895 (local address 127.5.91.190:46243)
0620 20:12:04.888225 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:04.888228 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:04.888954 (+ 726us) server_negotiation.cc:407] Connection header received
0620 20:12:04.890273 (+ 1319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:04.890276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:04.890336 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:04.890408 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:04.891681 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:04.892258 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:04.893519 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:04.893755 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:04.893844 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:04.896438 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:04.896442 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:04.896444 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:04.896467 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:04.897838 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:04.898390 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:04.898394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:04.898395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:04.898468 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:04.898788 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:04.898791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:04.898792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:04.898969 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:04.899061 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:04.899541 (+ 480us) server_negotiation.cc:299] Negotiation successful
0620 20:12:04.899681 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260620 20:12:04.900696 21794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:04.888380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.130:49895)
0620 20:12:04.888802 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:04.888842 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:04.890052 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:04.890589 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:04.890596 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:04.890943 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:04.891511 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:04.891526 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:04.892391 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:04.892394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:04.893369 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:04.893381 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:04.893898 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:04.893903 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:04.894064 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:04.894689 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:12:04.894706 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:04.896263 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:04.897952 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:04.897958 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:04.897973 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:04.898300 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:04.898554 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:04.898557 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:04.898559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:04.898694 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:04.899066 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:04.899073 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:04.899313 (+ 240us) client_negotiation.cc:769] Sending connection context
0620 20:12:04.899469 (+ 156us) client_negotiation.cc:240] Negotiation successful
0620 20:12:04.899852 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":116,"threads_started":1}
I20260620 20:12:04.901793 21792 heartbeater.cc:347] Connected to a master server at 127.5.91.190:46243
I20260620 20:12:04.902020 21792 heartbeater.cc:464] Registering TS with master...
I20260620 20:12:04.902546 21792 heartbeater.cc:511] Master 127.5.91.190:46243 requested a full tablet report, sending...
I20260620 20:12:04.903539 21464 ts_manager.cc:194] Registered new tserver with Master: 1da99bb317f4475a9bba1281755d974a (127.5.91.130:44031)
I20260620 20:12:04.904105 21464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:49895
W20260620 20:12:04.999823 21795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:05.000133 21795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:05.000196 21795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:05.005342 21795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:05.005455 21795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:05.005537 21795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:12:05.013060 21795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:33369
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=38181
--tserver_master_addrs=127.5.91.190:46243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:05.014274 21795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:05.016077 21795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:05.025925 21801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:05.025919 21803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:05.026023 21800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:05.026191 21795 server_base.cc:1034] running on GCE node
I20260620 20:12:05.026693 21795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:05.027316 21795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:05.028512 21795 hybrid_clock.cc:648] HybridClock initialized: now 1781986325028478 us; error 44 us; skew 500 ppm
Jun 20 20:12:05 dist-test-slave-l8h4 krb5kdc[20878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986325, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:05.031675 21795 init.cc:377] Logged in from keytab as kudu/127.5.91.131@KRBTEST.COM (short username kudu)
I20260620 20:12:05.032516 21795 webserver.cc:460] Webserver started at http://127.5.91.131:38181/ using document root <none> and password file <none>
I20260620 20:12:05.033033 21795 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:05.033109 21795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:05.037066 21795 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 20:12:05.039165 21810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:05.040398 21795 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:12:05.040501 21795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "c298adebc87e4fc09a967bb4f5e6fe79"
format_stamp: "Formatted at 2026-06-20 20:11:43 on dist-test-slave-l8h4"
server_key: "eae95c15021f0077ba6eed0b9faad7e3"
server_key_iv: "3e2c399ba0395542b9f2bb8ee4d15dcd"
server_key_version: "encryptionkey@0"
I20260620 20:12:05.041016 21795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:05.053925 21795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:05.057005 21795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:05.057186 21795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:05.057758 21795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:05.058588 21795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:12:05.058634 21795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:05.058686 21795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:12:05.058712 21795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:05.067948 21795 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:33369
I20260620 20:12:05.068029 21924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:33369 every 8 connection(s)
I20260620 20:12:05.068928 21795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:12:05.076913 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 21795
Jun 20 20:12:05 dist-test-slave-l8h4 krb5kdc[20878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986325, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:12:05.082985 21655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:05.070853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:39485 (local address 127.5.91.190:46243)
0620 20:12:05.071012 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:05.071016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:05.071785 (+ 769us) server_negotiation.cc:407] Connection header received
0620 20:12:05.072964 (+ 1179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:05.072967 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:05.073014 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:05.073121 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:05.074248 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.074799 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.076023 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.076245 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.076331 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:05.079116 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:05.079119 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:05.079121 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:05.079142 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:05.080620 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:05.081210 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:05.081213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:05.081215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:05.081303 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:05.081833 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:05.081837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:05.081839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:05.082085 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:05.082213 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:05.082730 (+ 517us) server_negotiation.cc:299] Negotiation successful
0620 20:12:05.082858 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59,"spinlock_wait_cycles":5888}
I20260620 20:12:05.084038 21927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:05.071201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.131:39485)
0620 20:12:05.071643 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:05.071684 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:05.072774 (+ 1090us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:05.073283 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:05.073291 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:05.073630 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:05.074097 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:05.074111 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.074906 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.074909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:05.075896 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:05.075906 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.076404 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.076407 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:05.076557 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:05.077199 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:12:05.077218 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:05.078950 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:05.080747 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:05.080752 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:05.080763 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:05.081103 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:05.081399 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:05.081401 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:05.081403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:05.081532 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:05.082241 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:05.082247 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:05.082477 (+ 230us) client_negotiation.cc:769] Sending connection context
0620 20:12:05.082657 (+ 180us) client_negotiation.cc:240] Negotiation successful
0620 20:12:05.083016 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":103,"threads_started":1}
I20260620 20:12:05.085243 21925 heartbeater.cc:347] Connected to a master server at 127.5.91.190:46243
I20260620 20:12:05.085546 21925 heartbeater.cc:464] Registering TS with master...
I20260620 20:12:05.086158 21925 heartbeater.cc:511] Master 127.5.91.190:46243 requested a full tablet report, sending...
I20260620 20:12:05.086231 21929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:05.077636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50334 (local address 127.5.91.190:46243)
0620 20:12:05.077884 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:05.077888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:05.077993 (+ 105us) server_negotiation.cc:407] Connection header received
0620 20:12:05.078108 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:05.078110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:05.078176 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:05.078270 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:05.079026 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.079696 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.080881 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.081218 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.081289 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:05.082652 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:05.082655 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:05.082658 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:05.082708 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:05.084492 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:05.084930 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:05.084952 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:05.084954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:05.085035 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:05.085379 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:05.085384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:05.085386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:05.085573 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:05.085746 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:05.085950 (+ 204us) server_negotiation.cc:299] Negotiation successful
0620 20:12:05.086060 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"spinlock_wait_cycles":31360,"thread_start_us":80,"threads_started":1}
I20260620 20:12:05.087180 21464 ts_manager.cc:194] Registered new tserver with Master: c298adebc87e4fc09a967bb4f5e6fe79 (127.5.91.131:33369)
I20260620 20:12:05.087776 21464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:39485
I20260620 20:12:05.087879 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 20:12:05.092008 21929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:05.088441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50336 (local address 127.5.91.190:46243)
0620 20:12:05.088548 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:05.088559 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:05.088749 (+ 190us) server_negotiation.cc:407] Connection header received
0620 20:12:05.088853 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:05.088856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:05.088958 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:05.089174 (+ 216us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:12:05.089373 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.090078 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.090931 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.091172 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.091250 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:05.091586 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:12:05.091702 (+ 116us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0620 20:12:05.091899 (+ 197us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50336: token expired
Metrics: {"server-negotiator.queue_time_us":40}
W20260620 20:12:05.092103 21929 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50336: token expired
W20260620 20:12:05.092435 21390 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.5.91.190:46243)
I20260620 20:12:05.099197 21929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:05.092764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50350 (local address 127.5.91.190:46243)
0620 20:12:05.092869 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:05.092873 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:05.092885 (+ 12us) server_negotiation.cc:407] Connection header received
0620 20:12:05.093020 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:05.093023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:05.093058 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:05.093131 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:05.093686 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.094177 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.094933 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.095148 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.095228 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:05.096452 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:05.096455 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:05.096458 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:05.096498 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:05.097774 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:05.098161 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:05.098164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:05.098166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:05.098237 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:05.098462 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:05.098465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:05.098467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:05.098628 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:05.098740 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:05.098938 (+ 198us) server_negotiation.cc:299] Negotiation successful
0620 20:12:05.099052 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 20:12:05.101480 21464 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50350:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:12:05.104648 21464 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:12:05.112483 21586 tablet_service.cc:1467] Processing CreateTablet for tablet eb163d04ce354758aea7ac518338647b (DEFAULT_TABLE table=test-table [id=bc205ca7053b4b8083bf9730e3752864]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:12:05.113107 21586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb163d04ce354758aea7ac518338647b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:05.115746 21544 tablet_bootstrap.cc:492] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363: Bootstrap starting.
I20260620 20:12:05.116544 21544 tablet_bootstrap.cc:654] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:05.117080 21660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:05.111561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:33369 (local address 127.0.0.1:38960)
0620 20:12:05.111669 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:05.111683 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:05.111756 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:05.112551 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:05.112554 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:05.112597 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:05.112819 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:05.112828 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.114457 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.114460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:05.115501 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:05.115511 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.116593 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.116597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:05.116711 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:05.116763 (+ 52us) client_negotiation.cc:769] Sending connection context
0620 20:12:05.116850 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 20:12:05.116937 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260620 20:12:05.117148 21934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:05.111730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38960 (local address 127.5.91.131:33369)
0620 20:12:05.112032 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:05.112046 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:05.112069 (+ 23us) server_negotiation.cc:407] Connection header received
0620 20:12:05.112149 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:05.112155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:05.112375 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:05.113074 (+ 699us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:05.113256 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.114356 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.115662 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.116460 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.116561 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:05.116703 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:05.116893 (+ 190us) server_negotiation.cc:299] Negotiation successful
0620 20:12:05.116994 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":110,"threads_started":1}
I20260620 20:12:05.117542 21935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:05.111949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:44031 (local address 127.0.0.1:53550)
0620 20:12:05.112636 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:05.112648 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:05.112839 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:05.113322 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:05.113324 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:05.113343 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:05.113442 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:05.113449 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.114904 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.114908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:05.115933 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:05.115944 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.117153 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.117157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:05.117256 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:05.117279 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 20:12:05.117355 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 20:12:05.117426 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":122,"threads_started":1}
I20260620 20:12:05.117633 21936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:05.112259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53550 (local address 127.5.91.130:44031)
0620 20:12:05.112588 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:05.112599 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:05.112809 (+ 210us) server_negotiation.cc:407] Connection header received
0620 20:12:05.112957 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:05.112962 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:05.113143 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:05.113288 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:05.113567 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.114790 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.116098 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.116867 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.116963 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:05.117082 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:05.117401 (+ 319us) server_negotiation.cc:299] Negotiation successful
0620 20:12:05.117496 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":112,"threads_started":1}
I20260620 20:12:05.118357 21544 tablet_bootstrap.cc:492] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363: No bootstrap required, opened a new log
I20260620 20:12:05.118436 21544 ts_tablet_manager.cc:1397] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 20:12:05.118861 21544 raft_consensus.cc:348] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } }
I20260620 20:12:05.118985 21544 raft_consensus.cc:374] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:05.119021 21544 raft_consensus.cc:729] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d92933d8838494bb957c61d72325363, State: Initialized, Role: FOLLOWER
I20260620 20:12:05.119140 21544 consensus_queue.cc:260] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } }
I20260620 20:12:05.119387 21544 ts_tablet_manager.cc:1428] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 20:12:05.120172 21858 tablet_service.cc:1467] Processing CreateTablet for tablet eb163d04ce354758aea7ac518338647b (DEFAULT_TABLE table=test-table [id=bc205ca7053b4b8083bf9730e3752864]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:12:05.120494 21725 tablet_service.cc:1467] Processing CreateTablet for tablet eb163d04ce354758aea7ac518338647b (DEFAULT_TABLE table=test-table [id=bc205ca7053b4b8083bf9730e3752864]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:12:05.121160 21858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb163d04ce354758aea7ac518338647b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:05.121446 21725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb163d04ce354758aea7ac518338647b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:05.127615 21939 tablet_bootstrap.cc:492] T eb163d04ce354758aea7ac518338647b P c298adebc87e4fc09a967bb4f5e6fe79: Bootstrap starting.
I20260620 20:12:05.127614 21938 tablet_bootstrap.cc:492] T eb163d04ce354758aea7ac518338647b P 1da99bb317f4475a9bba1281755d974a: Bootstrap starting.
I20260620 20:12:05.129489 21939 tablet_bootstrap.cc:654] T eb163d04ce354758aea7ac518338647b P c298adebc87e4fc09a967bb4f5e6fe79: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:05.129791 21938 tablet_bootstrap.cc:654] T eb163d04ce354758aea7ac518338647b P 1da99bb317f4475a9bba1281755d974a: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:05.130504 21938 log.cc:824] T eb163d04ce354758aea7ac518338647b P 1da99bb317f4475a9bba1281755d974a: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:05.130529 21939 log.cc:824] T eb163d04ce354758aea7ac518338647b P c298adebc87e4fc09a967bb4f5e6fe79: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:05.132436 21939 tablet_bootstrap.cc:492] T eb163d04ce354758aea7ac518338647b P c298adebc87e4fc09a967bb4f5e6fe79: No bootstrap required, opened a new log
I20260620 20:12:05.132429 21938 tablet_bootstrap.cc:492] T eb163d04ce354758aea7ac518338647b P 1da99bb317f4475a9bba1281755d974a: No bootstrap required, opened a new log
I20260620 20:12:05.132611 21939 ts_tablet_manager.cc:1397] T eb163d04ce354758aea7ac518338647b P c298adebc87e4fc09a967bb4f5e6fe79: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 20:12:05.132611 21938 ts_tablet_manager.cc:1397] T eb163d04ce354758aea7ac518338647b P 1da99bb317f4475a9bba1281755d974a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 20:12:05.136322 21939 raft_consensus.cc:348] T eb163d04ce354758aea7ac518338647b P c298adebc87e4fc09a967bb4f5e6fe79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } }
I20260620 20:12:05.136322 21938 raft_consensus.cc:348] T eb163d04ce354758aea7ac518338647b P 1da99bb317f4475a9bba1281755d974a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } }
I20260620 20:12:05.136595 21938 raft_consensus.cc:374] T eb163d04ce354758aea7ac518338647b P 1da99bb317f4475a9bba1281755d974a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:05.136595 21939 raft_consensus.cc:374] T eb163d04ce354758aea7ac518338647b P c298adebc87e4fc09a967bb4f5e6fe79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:05.136650 21939 raft_consensus.cc:729] T eb163d04ce354758aea7ac518338647b P c298adebc87e4fc09a967bb4f5e6fe79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c298adebc87e4fc09a967bb4f5e6fe79, State: Initialized, Role: FOLLOWER
I20260620 20:12:05.136657 21938 raft_consensus.cc:729] T eb163d04ce354758aea7ac518338647b P 1da99bb317f4475a9bba1281755d974a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1da99bb317f4475a9bba1281755d974a, State: Initialized, Role: FOLLOWER
I20260620 20:12:05.137105 21938 consensus_queue.cc:260] T eb163d04ce354758aea7ac518338647b P 1da99bb317f4475a9bba1281755d974a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } }
I20260620 20:12:05.137105 21939 consensus_queue.cc:260] T eb163d04ce354758aea7ac518338647b P c298adebc87e4fc09a967bb4f5e6fe79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } }
I20260620 20:12:05.137734 21925 heartbeater.cc:503] Master 127.5.91.190:46243 was elected leader, sending a full tablet report...
I20260620 20:12:05.138075 21938 ts_tablet_manager.cc:1428] T eb163d04ce354758aea7ac518338647b P 1da99bb317f4475a9bba1281755d974a: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260620 20:12:05.138075 21939 ts_tablet_manager.cc:1428] T eb163d04ce354758aea7ac518338647b P c298adebc87e4fc09a967bb4f5e6fe79: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260620 20:12:05.138628 21792 heartbeater.cc:503] Master 127.5.91.190:46243 was elected leader, sending a full tablet report...
W20260620 20:12:05.185928 21654 tablet.cc:2369] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:12:05.224845 21659 raft_consensus.cc:482] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:12:05.225044 21659 raft_consensus.cc:504] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } }
I20260620 20:12:05.226112 21659 leader_election.cc:288] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c298adebc87e4fc09a967bb4f5e6fe79 (127.5.91.131:33369), 1da99bb317f4475a9bba1281755d974a (127.5.91.130:44031)
I20260620 20:12:05.229959 21934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:05.226482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:36753 (local address 127.5.91.131:33369)
0620 20:12:05.226698 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:05.226704 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:05.226790 (+ 86us) server_negotiation.cc:407] Connection header received
0620 20:12:05.226937 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:05.226940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:05.226993 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:05.227061 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:05.227327 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.227850 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.228946 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.229345 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.229457 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:05.229509 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:05.229760 (+ 251us) server_negotiation.cc:299] Negotiation successful
0620 20:12:05.229834 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260620 20:12:05.229905 21945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:05.226356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:33369 (local address 127.5.91.129:36753)
0620 20:12:05.226708 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:05.226721 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:05.226826 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:05.227082 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:05.227084 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:05.227100 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:05.227225 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:05.227233 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.227949 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.227953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:05.228837 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:05.228846 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.229460 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.229463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:05.229612 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:05.229645 (+ 33us) client_negotiation.cc:769] Sending connection context
0620 20:12:05.229704 (+ 59us) client_negotiation.cc:240] Negotiation successful
0620 20:12:05.229757 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":77,"thread_start_us":114,"threads_started":1}
I20260620 20:12:05.229991 21936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:05.226516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:34259 (local address 127.5.91.130:44031)
0620 20:12:05.226679 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:05.226684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:05.226933 (+ 249us) server_negotiation.cc:407] Connection header received
0620 20:12:05.227033 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:05.227036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:05.227086 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:05.227159 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:05.227400 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.227876 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.228985 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.229369 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.229473 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:05.229521 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:05.229811 (+ 290us) server_negotiation.cc:299] Negotiation successful
0620 20:12:05.229881 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 20:12:05.230302 21944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:05.226372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:44031 (local address 127.5.91.129:34259)
0620 20:12:05.226828 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:05.226841 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:05.226945 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:05.227182 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:05.227185 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:05.227202 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:05.227294 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:05.227300 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.228006 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.228009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:05.228869 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:05.228879 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.229473 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.229477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:05.229670 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:05.229701 (+ 31us) client_negotiation.cc:769] Sending connection context
0620 20:12:05.229785 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 20:12:05.229848 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":99,"spinlock_wait_cycles":4352,"thread_start_us":103,"threads_started":1}
I20260620 20:12:05.230645 21879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb163d04ce354758aea7ac518338647b" candidate_uuid: "8d92933d8838494bb957c61d72325363" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" is_pre_election: true
I20260620 20:12:05.230868 21745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb163d04ce354758aea7ac518338647b" candidate_uuid: "8d92933d8838494bb957c61d72325363" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1da99bb317f4475a9bba1281755d974a" is_pre_election: true
I20260620 20:12:05.230909 21879 raft_consensus.cc:2454] T eb163d04ce354758aea7ac518338647b P c298adebc87e4fc09a967bb4f5e6fe79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d92933d8838494bb957c61d72325363 in term 0.
I20260620 20:12:05.231117 21745 raft_consensus.cc:2454] T eb163d04ce354758aea7ac518338647b P 1da99bb317f4475a9bba1281755d974a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d92933d8838494bb957c61d72325363 in term 0.
I20260620 20:12:05.231356 21541 leader_election.cc:302] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d92933d8838494bb957c61d72325363, c298adebc87e4fc09a967bb4f5e6fe79; no voters:
I20260620 20:12:05.231637 21659 raft_consensus.cc:2781] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:12:05.231725 21659 raft_consensus.cc:482] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:12:05.231776 21659 raft_consensus.cc:3037] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:05.232744 21659 raft_consensus.cc:504] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } }
I20260620 20:12:05.233065 21659 leader_election.cc:288] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 1 election: Requested vote from peers c298adebc87e4fc09a967bb4f5e6fe79 (127.5.91.131:33369), 1da99bb317f4475a9bba1281755d974a (127.5.91.130:44031)
I20260620 20:12:05.233381 21745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb163d04ce354758aea7ac518338647b" candidate_uuid: "8d92933d8838494bb957c61d72325363" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1da99bb317f4475a9bba1281755d974a"
I20260620 20:12:05.233493 21745 raft_consensus.cc:3037] T eb163d04ce354758aea7ac518338647b P 1da99bb317f4475a9bba1281755d974a [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:05.234261 21879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb163d04ce354758aea7ac518338647b" candidate_uuid: "8d92933d8838494bb957c61d72325363" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c298adebc87e4fc09a967bb4f5e6fe79"
I20260620 20:12:05.234372 21879 raft_consensus.cc:3037] T eb163d04ce354758aea7ac518338647b P c298adebc87e4fc09a967bb4f5e6fe79 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:05.234647 21745 raft_consensus.cc:2454] T eb163d04ce354758aea7ac518338647b P 1da99bb317f4475a9bba1281755d974a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d92933d8838494bb957c61d72325363 in term 1.
I20260620 20:12:05.234973 21538 leader_election.cc:302] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1da99bb317f4475a9bba1281755d974a, 8d92933d8838494bb957c61d72325363; no voters:
I20260620 20:12:05.235116 21659 raft_consensus.cc:2781] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:12:05.235289 21659 raft_consensus.cc:686] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 1 LEADER]: Becoming Leader. State: Replica: 8d92933d8838494bb957c61d72325363, State: Running, Role: LEADER
I20260620 20:12:05.235566 21879 raft_consensus.cc:2454] T eb163d04ce354758aea7ac518338647b P c298adebc87e4fc09a967bb4f5e6fe79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d92933d8838494bb957c61d72325363 in term 1.
I20260620 20:12:05.235528 21659 consensus_queue.cc:237] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } }
I20260620 20:12:05.238339 21463 catalog_manager.cc:5495] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d92933d8838494bb957c61d72325363 (127.5.91.129). New cstate: current_term: 1 leader_uuid: "8d92933d8838494bb957c61d72325363" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } health_report { overall_health: HEALTHY } } }
I20260620 20:12:05.271473 21948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:05.268048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50324 (local address 127.5.91.129:38321)
0620 20:12:05.268355 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:05.268361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:05.268372 (+ 11us) server_negotiation.cc:407] Connection header received
0620 20:12:05.268445 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:05.268447 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:05.268497 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:05.268603 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:12:05.268836 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.269346 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.270115 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:05.270351 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:05.270436 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:05.270675 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:12:05.271094 (+ 419us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:12:05.271187 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:05.271274 (+ 87us) server_negotiation.cc:299] Negotiation successful
0620 20:12:05.271329 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":111,"threads_started":1}
I20260620 20:12:05.275343 21879 raft_consensus.cc:1261] T eb163d04ce354758aea7ac518338647b P c298adebc87e4fc09a967bb4f5e6fe79 [term 1 FOLLOWER]: Refusing update from remote peer 8d92933d8838494bb957c61d72325363: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:12:05.275343 21745 raft_consensus.cc:1261] T eb163d04ce354758aea7ac518338647b P 1da99bb317f4475a9bba1281755d974a [term 1 FOLLOWER]: Refusing update from remote peer 8d92933d8838494bb957c61d72325363: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:12:05.276005 21946 consensus_queue.cc:1035] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:12:05.276232 21659 consensus_queue.cc:1035] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:12:05.282080 21949 mvcc.cc:205] Tried to move back new op lower bound from 7299015988321792000 to 7299015988167426048. Current Snapshot: MvccSnapshot[applied={T|T < 7299015988321792000}]
I20260620 20:12:05.285696 21950 mvcc.cc:205] Tried to move back new op lower bound from 7299015988321792000 to 7299015988167426048. Current Snapshot: MvccSnapshot[applied={T|T < 7299015988321792000 or (T in {7299015988321792000})}]
I20260620 20:12:05.290436 21464 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:12:05.290571 21464 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:12:05.292699 21464 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90: Sending DeleteTablet for 3 replicas of tablet eb163d04ce354758aea7ac518338647b
I20260620 20:12:05.293380 21586 tablet_service.cc:1514] Processing DeleteTablet for tablet eb163d04ce354758aea7ac518338647b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:12:05 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:50314
I20260620 20:12:05.293345 21858 tablet_service.cc:1514] Processing DeleteTablet for tablet eb163d04ce354758aea7ac518338647b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:12:05 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:38960
I20260620 20:12:05.293503 21725 tablet_service.cc:1514] Processing DeleteTablet for tablet eb163d04ce354758aea7ac518338647b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:12:05 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:53550
I20260620 20:12:05.293728 21958 tablet_replica.cc:330] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363: stopping tablet replica
I20260620 20:12:05.293846 21958 raft_consensus.cc:2229] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 1 LEADER]: Raft consensus shutting down.
I20260620 20:12:05.294013 21959 tablet_replica.cc:330] T eb163d04ce354758aea7ac518338647b P c298adebc87e4fc09a967bb4f5e6fe79: stopping tablet replica
I20260620 20:12:05.294065 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 21502
I20260620 20:12:05.294179 21958 raft_consensus.cc:2258] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:12:05.294909 21959 raft_consensus.cc:2229] T eb163d04ce354758aea7ac518338647b P c298adebc87e4fc09a967bb4f5e6fe79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:12:05.295212 21959 raft_consensus.cc:2258] T eb163d04ce354758aea7ac518338647b P c298adebc87e4fc09a967bb4f5e6fe79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:12:05.295962 21960 tablet_replica.cc:330] T eb163d04ce354758aea7ac518338647b P 1da99bb317f4475a9bba1281755d974a: stopping tablet replica
I20260620 20:12:05.296329 21960 raft_consensus.cc:2229] T eb163d04ce354758aea7ac518338647b P 1da99bb317f4475a9bba1281755d974a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:12:05.296624 21960 raft_consensus.cc:2258] T eb163d04ce354758aea7ac518338647b P 1da99bb317f4475a9bba1281755d974a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:12:05.296824 21959 ts_tablet_manager.cc:1905] T eb163d04ce354758aea7ac518338647b P c298adebc87e4fc09a967bb4f5e6fe79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:12:05.297948 21960 ts_tablet_manager.cc:1905] T eb163d04ce354758aea7ac518338647b P 1da99bb317f4475a9bba1281755d974a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:12:05.299485 21959 ts_tablet_manager.cc:1918] T eb163d04ce354758aea7ac518338647b P c298adebc87e4fc09a967bb4f5e6fe79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:12:05.299582 21959 log.cc:1196] T eb163d04ce354758aea7ac518338647b P c298adebc87e4fc09a967bb4f5e6fe79: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/eb163d04ce354758aea7ac518338647b
I20260620 20:12:05.299919 21959 ts_tablet_manager.cc:1939] T eb163d04ce354758aea7ac518338647b P c298adebc87e4fc09a967bb4f5e6fe79: Deleting consensus metadata
I20260620 20:12:05.300500 21960 ts_tablet_manager.cc:1918] T eb163d04ce354758aea7ac518338647b P 1da99bb317f4475a9bba1281755d974a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260620 20:12:05.300536 21449 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.91.129:38321 (error 108)
I20260620 20:12:05.300614 21960 log.cc:1196] T eb163d04ce354758aea7ac518338647b P 1da99bb317f4475a9bba1281755d974a: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/wal/wals/eb163d04ce354758aea7ac518338647b
I20260620 20:12:05.300840 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 21658
I20260620 20:12:05.300915 21960 ts_tablet_manager.cc:1939] T eb163d04ce354758aea7ac518338647b P 1da99bb317f4475a9bba1281755d974a: Deleting consensus metadata
I20260620 20:12:05.301195 21450 catalog_manager.cc:4841] TS c298adebc87e4fc09a967bb4f5e6fe79 (127.5.91.131:33369): tablet eb163d04ce354758aea7ac518338647b (table test-table [id=bc205ca7053b4b8083bf9730e3752864]) successfully deleted
I20260620 20:12:05.302911 21935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:05.302671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:38321 (local address 127.0.0.1:50338)
0620 20:12:05.302783 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:05.302842 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.129:38321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260620 20:12:05.303089 21449 catalog_manager.cc:4584] TS 8d92933d8838494bb957c61d72325363 (127.5.91.129:38321): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eb163d04ce354758aea7ac518338647b: Network error: Client connection negotiation failed: client connection to 127.5.91.129:38321: connect: Connection refused (error 111)
I20260620 20:12:05.305812 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 21795
I20260620 20:12:05.306293 21935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:05.306030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:44031 (local address 127.0.0.1:53560)
0620 20:12:05.306181 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:05.306222 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.130:44031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260620 20:12:05.310334 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 21430
I20260620 20:12:25.315603 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:46243
--webserver_interface=127.5.91.190
--webserver_port=37723
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:46243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:12:25.448016 21962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:25.448455 21962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:25.448525 21962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:25.454352 21962 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 20:12:25.454445 21962 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 20:12:25.454473 21962 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 20:12:25.454494 21962 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:12:25.454521 21962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:25.454550 21962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:25.454581 21962 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:12:25.454607 21962 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:12:25.460763 21962 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:46243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:46243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=37723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:25.461921 21962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:25.463001 21962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:25.471774 21968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:25.471774 21970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:25.472163 21962 server_base.cc:1034] running on GCE node
W20260620 20:12:25.472728 21967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:25.473541 21962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:25.474812 21962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:25.475994 21962 hybrid_clock.cc:648] HybridClock initialized: now 1781986345475970 us; error 41 us; skew 500 ppm
Jun 20 20:12:25 dist-test-slave-l8h4 krb5kdc[20878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986345, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:25.478999 21962 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:12:25.479987 21962 webserver.cc:460] Webserver started at http://127.5.91.190:37723/ using document root <none> and password file <none>
I20260620 20:12:25.480547 21962 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:25.480631 21962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:25.484790 21962 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.001s
I20260620 20:12:25.487018 21977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:25.488248 21962 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 20:12:25.488361 21962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "f6478905ca1f4e50b944e5274c384f90"
format_stamp: "Formatted at 2026-06-20 20:11:43 on dist-test-slave-l8h4"
server_key: "e71e575c547b495726c25ec2bd78000d"
server_key_iv: "4bdbda48ef9ac917c0e28347a60d7587"
server_key_version: "encryptionkey@0"
I20260620 20:12:25.488903 21962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:25.505681 21962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:25.509192 21962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:25.509537 21962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:25.518927 21962 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:46243
I20260620 20:12:25.519212 22030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:46243 every 8 connection(s)
I20260620 20:12:25.520160 21962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:12:25.523806 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 21962
I20260620 20:12:25.524255 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:38321
--local_ip_for_outbound_sockets=127.5.91.129
--tserver_master_addrs=127.5.91.190:46243
--webserver_port=36389
--webserver_interface=127.5.91.129
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 20:12:25.525185 22031 sys_catalog.cc:263] Verifying existing consensus state
I20260620 20:12:25.527977 22031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90: Bootstrap starting.
I20260620 20:12:25.546005 22031 log.cc:824] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:25.557696 22031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 20:12:25.558156 22031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90: Bootstrap complete.
I20260620 20:12:25.562206 22031 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6478905ca1f4e50b944e5274c384f90" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46243 } }
I20260620 20:12:25.562628 22031 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6478905ca1f4e50b944e5274c384f90, State: Initialized, Role: FOLLOWER
I20260620 20:12:25.563148 22031 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6478905ca1f4e50b944e5274c384f90" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46243 } }
I20260620 20:12:25.563282 22031 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:12:25.563362 22031 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:12:25.563457 22031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 2 FOLLOWER]: Advancing to term 3
I20260620 20:12:25.565375 22031 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6478905ca1f4e50b944e5274c384f90" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46243 } }
I20260620 20:12:25.565711 22031 leader_election.cc:302] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6478905ca1f4e50b944e5274c384f90; no voters:
I20260620 20:12:25.566016 22031 leader_election.cc:288] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260620 20:12:25.566126 22035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 3 FOLLOWER]: Leader election won for term 3
I20260620 20:12:25.566344 22035 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [term 3 LEADER]: Becoming Leader. State: Replica: f6478905ca1f4e50b944e5274c384f90, State: Running, Role: LEADER
I20260620 20:12:25.566672 22035 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6478905ca1f4e50b944e5274c384f90" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46243 } }
I20260620 20:12:25.567302 22031 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:12:25.568195 22037 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6478905ca1f4e50b944e5274c384f90. Latest consensus state: current_term: 3 leader_uuid: "f6478905ca1f4e50b944e5274c384f90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6478905ca1f4e50b944e5274c384f90" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46243 } } }
I20260620 20:12:25.568324 22037 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:25.568605 22036 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "f6478905ca1f4e50b944e5274c384f90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6478905ca1f4e50b944e5274c384f90" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46243 } } }
I20260620 20:12:25.568678 22036 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90 [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:25.568691 22044 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:12:25.573854 22044 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:12:25.574530 22044 catalog_manager.cc:1259] Loaded cluster ID: 6f3b3f9eeb994d8384ed0ab5c21539d8
I20260620 20:12:25.574590 22044 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:12:25.576360 22044 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:12:25.576974 22044 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90: Loaded TSK: 0
I20260620 20:12:25.577040 22044 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f6478905ca1f4e50b944e5274c384f90: Loaded TSK: 1
F20260620 20:12:25.577647 22044 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f8777a8e9ad google::LogMessage::Fail() at ??:0
@ 0x7f8777a9188a google::LogMessage::SendToLog() at ??:0
@ 0x7f8777a8e4de google::LogMessage::Flush() at ??:0
@ 0x7f8777a8fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f877a75218d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f877a74af0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f87823f37bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f87823cd99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f87823ce178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f878240fdf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f878242e82b std::function<>::operator()() at ??:0
@ 0x7f87823cdb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f87823cef3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f87823cb579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f878240f640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f8779ce03d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f8779ce0c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f8779ce23f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f8779cd1512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f877b3156db start_thread at ??:0
@ 0x7f877658c61f clone at ??:0
W20260620 20:12:25.681905 22033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:25.682250 22033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:25.682329 22033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:25.686990 22033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:25.687067 22033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:25.687165 22033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:12:25.692745 22033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:38321
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=36389
--tserver_master_addrs=127.5.91.190:46243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:25.693848 22033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:25.694923 22033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:25.704756 22078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:25.704756 22079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:25.704758 22081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:25.705257 22033 server_base.cc:1034] running on GCE node
I20260620 20:12:25.705765 22033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:25.706444 22033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:25.707614 22033 hybrid_clock.cc:648] HybridClock initialized: now 1781986345707596 us; error 37 us; skew 500 ppm
Jun 20 20:12:25 dist-test-slave-l8h4 krb5kdc[20878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986345, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:25.710685 22033 init.cc:377] Logged in from keytab as kudu/127.5.91.129@KRBTEST.COM (short username kudu)
I20260620 20:12:25.711596 22033 webserver.cc:460] Webserver started at http://127.5.91.129:36389/ using document root <none> and password file <none>
I20260620 20:12:25.712117 22033 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:25.712180 22033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:25.715994 22033 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:12:25.718122 22088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:25.720000 22033 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 20:12:25.720105 22033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "8d92933d8838494bb957c61d72325363"
format_stamp: "Formatted at 2026-06-20 20:11:43 on dist-test-slave-l8h4"
server_key: "fe303842c202b2d9889ca90b71a35879"
server_key_iv: "37e16f5aaf9372e3e7c3830a3901a78e"
server_key_version: "encryptionkey@0"
I20260620 20:12:25.720698 22033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:25.751747 22033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:25.754987 22033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:25.755169 22033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:25.755767 22033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:25.756845 22095 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 20:12:25.761147 22033 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 20:12:25.761216 22033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260620 20:12:25.761270 22033 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 20:12:25.763329 22033 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 20:12:25.763388 22033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260620 20:12:25.763666 22095 tablet_bootstrap.cc:492] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363: Bootstrap starting.
I20260620 20:12:25.774048 22033 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:38321
I20260620 20:12:25.774302 22203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:38321 every 8 connection(s)
I20260620 20:12:25.775995 22033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:12:25.782464 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 22033
I20260620 20:12:25.782944 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:44031
--local_ip_for_outbound_sockets=127.5.91.130
--tserver_master_addrs=127.5.91.190:46243
--webserver_port=35985
--webserver_interface=127.5.91.130
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 20:12:25.786347 22095 log.cc:824] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:25.795907 22095 tablet_bootstrap.cc:492] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 20:12:25.796344 22095 tablet_bootstrap.cc:492] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363: Bootstrap complete.
I20260620 20:12:25.797574 22095 ts_tablet_manager.cc:1397] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363: Time spent bootstrapping tablet: real 0.034s user 0.025s sys 0.004s
I20260620 20:12:25.800946 22095 raft_consensus.cc:348] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } }
I20260620 20:12:25.801517 22095 raft_consensus.cc:729] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d92933d8838494bb957c61d72325363, State: Initialized, Role: FOLLOWER
I20260620 20:12:25.802048 22095 consensus_queue.cc:260] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } }
I20260620 20:12:25.803109 22095 ts_tablet_manager.cc:1428] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
W20260620 20:12:25.812979 22206 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:12:25.780556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.129:39083)
0620 20:12:25.784734 (+ 4178us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:25.784811 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:25.785951 (+ 1140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:25.812256 (+ 26305us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1543,"thread_start_us":190,"threads_started":1}
W20260620 20:12:25.813601 22204 heartbeater.cc:644] Failed to heartbeat to 127.5.91.190:46243 (0 consecutive failures): Network error: Failed to ping master at 127.5.91.190:46243: Client connection negotiation failed: client connection to 127.5.91.190:46243: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 20:12:25.814366 22206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:25.814078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.129:46137)
0620 20:12:25.814229 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:25.814301 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260620 20:12:25.814996 22206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:25.814786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.129:42533)
0620 20:12:25.814892 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:25.814940 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260620 20:12:25.815215 22204 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 20:12:25.900280 22207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:25.900625 22207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:25.900703 22207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:25.905504 22207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:25.905589 22207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:25.905655 22207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:12:25.911796 22207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:44031
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=35985
--tserver_master_addrs=127.5.91.190:46243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:25.912842 22207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:25.913790 22207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:25.924197 22214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:25.924250 22215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:25.924190 22217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:25.924686 22207 server_base.cc:1034] running on GCE node
I20260620 20:12:25.925092 22207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:25.925778 22207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:25.927001 22207 hybrid_clock.cc:648] HybridClock initialized: now 1781986345927008 us; error 96 us; skew 500 ppm
Jun 20 20:12:25 dist-test-slave-l8h4 krb5kdc[20878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986345, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:25.930522 22207 init.cc:377] Logged in from keytab as kudu/127.5.91.130@KRBTEST.COM (short username kudu)
I20260620 20:12:25.931643 22207 webserver.cc:460] Webserver started at http://127.5.91.130:35985/ using document root <none> and password file <none>
I20260620 20:12:25.932226 22207 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:25.932299 22207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:25.936223 22207 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 20:12:25.938680 22224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:25.940398 22207 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:12:25.940538 22207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "1da99bb317f4475a9bba1281755d974a"
format_stamp: "Formatted at 2026-06-20 20:11:43 on dist-test-slave-l8h4"
server_key: "981ea23d7c9452b34387a8936b699a7c"
server_key_iv: "c9f01cbad968e9920f491b8267a036d2"
server_key_version: "encryptionkey@0"
I20260620 20:12:25.941534 22207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:25.968204 22207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:25.971604 22207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:25.971786 22207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:25.972384 22207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:25.973214 22207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:12:25.973261 22207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:25.973315 22207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:12:25.973343 22207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:25.983695 22207 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:44031
I20260620 20:12:25.983726 22338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:44031 every 8 connection(s)
I20260620 20:12:25.984697 22207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:12:25.989005 22341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:25.986901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.130:37161)
0620 20:12:25.987756 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:25.987895 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":124,"threads_started":1}
W20260620 20:12:25.989538 22339 heartbeater.cc:644] Failed to heartbeat to 127.5.91.190:46243 (0 consecutive failures): Network error: Failed to ping master at 127.5.91.190:46243: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
I20260620 20:12:25.990260 22341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:25.989923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.130:50061)
0620 20:12:25.990107 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:25.990197 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260620 20:12:25.991376 22341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:25.991010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.130:46295)
0620 20:12:25.991191 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:25.991278 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260620 20:12:25.991673 22339 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 20:12:25.992156 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 22207
I20260620 20:12:25.992532 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:33369
--local_ip_for_outbound_sockets=127.5.91.131
--tserver_master_addrs=127.5.91.190:46243
--webserver_port=38181
--webserver_interface=127.5.91.131
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:12:26.116041 22342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:26.116320 22342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:26.116364 22342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:26.120466 22342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:26.120527 22342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:26.120590 22342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:12:26.125924 22342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:33369
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=38181
--tserver_master_addrs=127.5.91.190:46243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:26.126957 22342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:26.127904 22342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:26.137450 22350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:26.137423 22347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:26.137418 22348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:26.137622 22342 server_base.cc:1034] running on GCE node
I20260620 20:12:26.138185 22342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:26.138866 22342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:26.140033 22342 hybrid_clock.cc:648] HybridClock initialized: now 1781986346139993 us; error 55 us; skew 500 ppm
Jun 20 20:12:26 dist-test-slave-l8h4 krb5kdc[20878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986346, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:26.143330 22342 init.cc:377] Logged in from keytab as kudu/127.5.91.131@KRBTEST.COM (short username kudu)
I20260620 20:12:26.144320 22342 webserver.cc:460] Webserver started at http://127.5.91.131:38181/ using document root <none> and password file <none>
I20260620 20:12:26.144865 22342 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:26.144963 22342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:26.148795 22342 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260620 20:12:26.150892 22357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:26.152355 22342 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260620 20:12:26.152468 22342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "c298adebc87e4fc09a967bb4f5e6fe79"
format_stamp: "Formatted at 2026-06-20 20:11:43 on dist-test-slave-l8h4"
server_key: "eae95c15021f0077ba6eed0b9faad7e3"
server_key_iv: "3e2c399ba0395542b9f2bb8ee4d15dcd"
server_key_version: "encryptionkey@0"
I20260620 20:12:26.153025 22342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:26.185916 22342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:26.189102 22342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:26.189280 22342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:26.189859 22342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:26.190737 22342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:12:26.190784 22342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:26.190837 22342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:12:26.190866 22342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:26.199712 22342 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:33369
I20260620 20:12:26.199743 22471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:33369 every 8 connection(s)
I20260620 20:12:26.200747 22342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:12:26.204675 22474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:26.202988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.131:39557)
0620 20:12:26.203478 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:26.203978 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":118,"threads_started":1}
W20260620 20:12:26.205351 22472 heartbeater.cc:644] Failed to heartbeat to 127.5.91.190:46243 (0 consecutive failures): Network error: Failed to ping master at 127.5.91.190:46243: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
I20260620 20:12:26.206100 22474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:26.205867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.131:60563)
0620 20:12:26.205995 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:26.206043 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260620 20:12:26.206887 22474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:26.206663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.131:51367)
0620 20:12:26.206791 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:26.206836 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260620 20:12:26.207206 22472 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 20:12:26.210476 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 22342
I20260620 20:12:26.816210 22476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:26.815770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.129:43479)
0620 20:12:26.816042 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:26.816100 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":71,"threads_started":1}
I20260620 20:12:26.992728 22477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:26.992316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.130:55351)
0620 20:12:26.992558 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:26.992619 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":63,"threads_started":1}
I20260620 20:12:27.013178 22478 raft_consensus.cc:482] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:12:27.013365 22478 raft_consensus.cc:504] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } }
I20260620 20:12:27.014475 22478 leader_election.cc:288] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c298adebc87e4fc09a967bb4f5e6fe79 (127.5.91.131:33369), 1da99bb317f4475a9bba1281755d974a (127.5.91.130:44031)
Jun 20 20:12:27 dist-test-slave-l8h4 krb5kdc[20878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986345, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.130@KRBTEST.COM
Jun 20 20:12:27 dist-test-slave-l8h4 krb5kdc[20878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986345, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.131@KRBTEST.COM
I20260620 20:12:27.032719 22476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:27.014696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:33369 (local address 127.5.91.129:49993)
0620 20:12:27.014828 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:27.014843 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:27.014942 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:27.016667 (+ 1725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:27.016673 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:27.017258 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:27.017772 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:27.017788 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:27.019460 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:27.019463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:27.020432 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:27.020441 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:27.021121 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:27.021125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:27.021327 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:27.026250 (+ 4923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:12:27.026274 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:27.028259 (+ 1985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:27.030747 (+ 2488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:27.030750 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:27.030752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:27.030930 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:27.031240 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:27.031243 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:27.031245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:27.031329 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:27.031888 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:27.031891 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:27.032044 (+ 153us) client_negotiation.cc:769] Sending connection context
0620 20:12:27.032212 (+ 168us) client_negotiation.cc:240] Negotiation successful
0620 20:12:27.032519 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":105,"spinlock_wait_cycles":41984}
I20260620 20:12:27.032752 22480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:27.014993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:49993 (local address 127.5.91.131:33369)
0620 20:12:27.015300 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:27.015311 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:27.015344 (+ 33us) server_negotiation.cc:407] Connection header received
0620 20:12:27.015984 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:27.015989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:27.016326 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:27.016591 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:27.018045 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:27.019299 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:27.020547 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:27.021009 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:27.021160 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:27.028474 (+ 7314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:27.028479 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:27.028491 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:27.028530 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:27.030631 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:27.031033 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:27.031038 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:27.031043 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:27.031153 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:27.031407 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:27.031411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:27.031412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:27.031784 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:27.031901 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:27.032341 (+ 440us) server_negotiation.cc:299] Negotiation successful
0620 20:12:27.032603 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":60,"threads_started":1}
I20260620 20:12:27.033022 22479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:27.014745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:44031 (local address 127.5.91.129:54233)
0620 20:12:27.015076 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:27.015087 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:27.015175 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:27.016666 (+ 1491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:27.016672 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:27.017324 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:27.017907 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:27.017929 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:27.019353 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:27.019357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:27.020426 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:27.020437 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:27.021123 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:27.021127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:27.021327 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:27.026244 (+ 4917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:12:27.026263 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:27.027925 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:27.030405 (+ 2480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:27.030411 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:27.030425 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:27.030735 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:27.031050 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:27.031054 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:27.031055 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:27.031186 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:27.031815 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:27.031821 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:27.032044 (+ 223us) client_negotiation.cc:769] Sending connection context
0620 20:12:27.032387 (+ 343us) client_negotiation.cc:240] Negotiation successful
0620 20:12:27.032679 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":70,"spinlock_wait_cycles":9856,"thread_start_us":74,"threads_started":1}
I20260620 20:12:27.033568 22481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:27.014929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:54233 (local address 127.5.91.130:44031)
0620 20:12:27.015327 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:27.015345 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:27.015373 (+ 28us) server_negotiation.cc:407] Connection header received
0620 20:12:27.016016 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:27.016021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:27.016325 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:27.016591 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:27.018172 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:27.019203 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:27.020546 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:27.021008 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:27.021162 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:27.028198 (+ 7036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:27.028205 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:27.028218 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:27.028251 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:27.030282 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:27.030826 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:27.030833 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:27.030838 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:27.030961 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:27.031275 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:27.031278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:27.031280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:27.031703 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:27.031821 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:27.032681 (+ 860us) server_negotiation.cc:299] Negotiation successful
0620 20:12:27.033433 (+ 752us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":123,"threads_started":1}
W20260620 20:12:27.035832 22092 leader_election.cc:341] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c298adebc87e4fc09a967bb4f5e6fe79 (127.5.91.131:33369): Not found: Tablet not found: eb163d04ce354758aea7ac518338647b
W20260620 20:12:27.036636 22089 leader_election.cc:341] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1da99bb317f4475a9bba1281755d974a (127.5.91.130:44031): Not found: Tablet not found: eb163d04ce354758aea7ac518338647b
I20260620 20:12:27.036738 22089 leader_election.cc:302] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d92933d8838494bb957c61d72325363; no voters: 1da99bb317f4475a9bba1281755d974a, c298adebc87e4fc09a967bb4f5e6fe79
I20260620 20:12:27.036993 22478 raft_consensus.cc:2726] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 20:12:27.208177 22482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:27.207762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.131:36519)
0620 20:12:27.208025 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:27.208083 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":75,"threads_started":1}
I20260620 20:12:27.817521 22483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:27.817060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.129:39689)
0620 20:12:27.817343 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:27.817420 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":79,"threads_started":1}
I20260620 20:12:27.993952 22484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:27.993493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.130:34777)
0620 20:12:27.993802 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:27.993862 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":84,"threads_started":1}
I20260620 20:12:28.209360 22485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:28.208952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.131:46945)
0620 20:12:28.209204 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:28.209257 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":79,"threads_started":1}
I20260620 20:12:28.818759 22486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:28.818348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.129:57257)
0620 20:12:28.818610 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:28.818667 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":74,"threads_started":1}
I20260620 20:12:28.914701 22487 raft_consensus.cc:482] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:12:28.914845 22487 raft_consensus.cc:504] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } }
I20260620 20:12:28.915426 22487 leader_election.cc:288] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c298adebc87e4fc09a967bb4f5e6fe79 (127.5.91.131:33369), 1da99bb317f4475a9bba1281755d974a (127.5.91.130:44031)
W20260620 20:12:28.916229 22089 leader_election.cc:341] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1da99bb317f4475a9bba1281755d974a (127.5.91.130:44031): Not found: Tablet not found: eb163d04ce354758aea7ac518338647b
W20260620 20:12:28.916354 22092 leader_election.cc:341] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c298adebc87e4fc09a967bb4f5e6fe79 (127.5.91.131:33369): Not found: Tablet not found: eb163d04ce354758aea7ac518338647b
I20260620 20:12:28.916407 22092 leader_election.cc:302] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d92933d8838494bb957c61d72325363; no voters: 1da99bb317f4475a9bba1281755d974a, c298adebc87e4fc09a967bb4f5e6fe79
I20260620 20:12:28.916579 22487 raft_consensus.cc:2726] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 20:12:28.995234 22488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:28.994787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.130:47675)
0620 20:12:28.995059 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:28.995125 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":88,"threads_started":1}
I20260620 20:12:29.210697 22489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:29.210260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.131:58717)
0620 20:12:29.210533 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:29.210594 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":89,"threads_started":1}
I20260620 20:12:29.819998 22490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:29.819552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.129:34905)
0620 20:12:29.819845 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:29.819905 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":88,"threads_started":1}
I20260620 20:12:29.996600 22491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:29.996139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.130:34283)
0620 20:12:29.996436 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:29.996498 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":75,"threads_started":1}
I20260620 20:12:30.211833 22492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:30.211444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.131:51305)
0620 20:12:30.211690 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:30.211744 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":76,"threads_started":1}
I20260620 20:12:30.821867 22493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:30.821356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.129:41477)
0620 20:12:30.821714 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:30.821775 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":107,"threads_started":1}
I20260620 20:12:30.997959 22494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:30.997430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.130:53587)
0620 20:12:30.997804 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:30.997865 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":87,"threads_started":1}
I20260620 20:12:31.213220 22495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:31.212749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.131:41119)
0620 20:12:31.213056 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:31.213121 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":78,"threads_started":1}
I20260620 20:12:31.348989 22496 raft_consensus.cc:482] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:12:31.349279 22496 raft_consensus.cc:504] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } }
I20260620 20:12:31.349804 22496 leader_election.cc:288] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c298adebc87e4fc09a967bb4f5e6fe79 (127.5.91.131:33369), 1da99bb317f4475a9bba1281755d974a (127.5.91.130:44031)
W20260620 20:12:31.350872 22092 leader_election.cc:341] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c298adebc87e4fc09a967bb4f5e6fe79 (127.5.91.131:33369): Not found: Tablet not found: eb163d04ce354758aea7ac518338647b
W20260620 20:12:31.350972 22089 leader_election.cc:341] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1da99bb317f4475a9bba1281755d974a (127.5.91.130:44031): Not found: Tablet not found: eb163d04ce354758aea7ac518338647b
I20260620 20:12:31.351107 22089 leader_election.cc:302] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d92933d8838494bb957c61d72325363; no voters: 1da99bb317f4475a9bba1281755d974a, c298adebc87e4fc09a967bb4f5e6fe79
I20260620 20:12:31.351261 22496 raft_consensus.cc:2726] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 20:12:31.823033 22497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:31.822579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.129:34343)
0620 20:12:31.822888 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:31.822940 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":153,"threads_started":1}
I20260620 20:12:31.999009 22498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:31.998636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.130:45977)
0620 20:12:31.998868 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:31.998921 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":76,"threads_started":1}
I20260620 20:12:32.214483 22499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:32.214009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.131:44781)
0620 20:12:32.214310 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:32.214370 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":127,"threads_started":1}
I20260620 20:12:32.824141 22500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:32.823702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.129:55169)
0620 20:12:32.823959 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:32.824036 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":81,"threads_started":1}
I20260620 20:12:33.000203 22501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:32.999715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.130:50301)
0620 20:12:33.000049 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:33.000105 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":80,"threads_started":1}
I20260620 20:12:33.215757 22502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:33.215265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.131:48713)
0620 20:12:33.215575 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:33.215641 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":72,"threads_started":1}
I20260620 20:12:33.825268 22503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:33.824847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.129:35375)
0620 20:12:33.825119 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:33.825174 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":82,"threads_started":1}
I20260620 20:12:34.001497 22504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:34.000993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.130:34795)
0620 20:12:34.001337 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:34.001393 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":77,"threads_started":1}
I20260620 20:12:34.150791 22505 raft_consensus.cc:482] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:12:34.150933 22505 raft_consensus.cc:504] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } }
I20260620 20:12:34.151346 22505 leader_election.cc:288] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c298adebc87e4fc09a967bb4f5e6fe79 (127.5.91.131:33369), 1da99bb317f4475a9bba1281755d974a (127.5.91.130:44031)
W20260620 20:12:34.152164 22092 leader_election.cc:341] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c298adebc87e4fc09a967bb4f5e6fe79 (127.5.91.131:33369): Not found: Tablet not found: eb163d04ce354758aea7ac518338647b
W20260620 20:12:34.152274 22089 leader_election.cc:341] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1da99bb317f4475a9bba1281755d974a (127.5.91.130:44031): Not found: Tablet not found: eb163d04ce354758aea7ac518338647b
I20260620 20:12:34.152340 22089 leader_election.cc:302] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d92933d8838494bb957c61d72325363; no voters: 1da99bb317f4475a9bba1281755d974a, c298adebc87e4fc09a967bb4f5e6fe79
I20260620 20:12:34.152530 22505 raft_consensus.cc:2726] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 20:12:34.216986 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:34.216552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.131:39507)
0620 20:12:34.216829 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:34.216887 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":108,"threads_started":1}
I20260620 20:12:34.826412 22507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:34.825981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.129:39679)
0620 20:12:34.826260 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:34.826317 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":89,"threads_started":1}
I20260620 20:12:35.002831 22508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:35.002434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.130:51549)
0620 20:12:35.002686 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:35.002735 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":75,"threads_started":1}
I20260620 20:12:35.218123 22509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:35.217684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.131:41013)
0620 20:12:35.217953 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:35.218017 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":81,"threads_started":1}
I20260620 20:12:35.827600 22510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:35.827210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.129:41719)
0620 20:12:35.827449 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:35.827504 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":73,"threads_started":1}
I20260620 20:12:36.004138 22511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:36.003666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.130:46809)
0620 20:12:36.003937 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:36.004040 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":77,"threads_started":1}
I20260620 20:12:36.219322 22512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:36.218869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.131:53333)
0620 20:12:36.219163 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:36.219224 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":90,"threads_started":1}
I20260620 20:12:36.829030 22513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:36.828366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.129:48785)
0620 20:12:36.828670 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:36.828884 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":86,"threads_started":1}
I20260620 20:12:37.005632 22514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:37.005006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.130:40385)
0620 20:12:37.005444 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:37.005528 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":77,"threads_started":1}
I20260620 20:12:37.220854 22515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:37.220191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.131:33419)
0620 20:12:37.220601 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:37.220710 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":91,"threads_started":1}
I20260620 20:12:37.830435 22516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:37.829965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.129:47069)
0620 20:12:37.830278 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:37.830331 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":86,"threads_started":1}
I20260620 20:12:38.006907 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:38.006475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.130:35181)
0620 20:12:38.006752 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:38.006802 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":89,"threads_started":1}
I20260620 20:12:38.222333 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:38.221791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.131:41307)
0620 20:12:38.222095 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:38.222205 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":98,"threads_started":1}
I20260620 20:12:38.229578 22519 raft_consensus.cc:482] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:12:38.229708 22519 raft_consensus.cc:504] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c298adebc87e4fc09a967bb4f5e6fe79" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 33369 } } peers { permanent_uuid: "1da99bb317f4475a9bba1281755d974a" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 44031 } } peers { permanent_uuid: "8d92933d8838494bb957c61d72325363" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38321 } }
I20260620 20:12:38.230068 22519 leader_election.cc:288] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c298adebc87e4fc09a967bb4f5e6fe79 (127.5.91.131:33369), 1da99bb317f4475a9bba1281755d974a (127.5.91.130:44031)
W20260620 20:12:38.230798 22092 leader_election.cc:341] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c298adebc87e4fc09a967bb4f5e6fe79 (127.5.91.131:33369): Not found: Tablet not found: eb163d04ce354758aea7ac518338647b
W20260620 20:12:38.230883 22089 leader_election.cc:341] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1da99bb317f4475a9bba1281755d974a (127.5.91.130:44031): Not found: Tablet not found: eb163d04ce354758aea7ac518338647b
I20260620 20:12:38.230962 22089 leader_election.cc:302] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d92933d8838494bb957c61d72325363; no voters: 1da99bb317f4475a9bba1281755d974a, c298adebc87e4fc09a967bb4f5e6fe79
I20260620 20:12:38.231202 22519 raft_consensus.cc:2726] T eb163d04ce354758aea7ac518338647b P 8d92933d8838494bb957c61d72325363 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 20:12:38.831712 22520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:38.831276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.129:38599)
0620 20:12:38.831535 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:38.831618 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":76,"threads_started":1}
I20260620 20:12:39.008334 22521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:39.007863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.130:48593)
0620 20:12:39.008171 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:39.008235 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":73,"threads_started":1}
I20260620 20:12:39.223508 22522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:39.223103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.131:33359)
0620 20:12:39.223357 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:39.223411 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":80,"threads_started":1}
I20260620 20:12:39.832937 22523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:39.832486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.129:47523)
0620 20:12:39.832782 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:39.832845 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":80,"threads_started":1}
I20260620 20:12:40.010032 22524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:40.009466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.130:42213)
0620 20:12:40.009827 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:40.009893 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":118,"threads_started":1}
I20260620 20:12:40.224823 22525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:40.224354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.131:38437)
0620 20:12:40.224635 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:40.224695 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":93,"threads_started":1}
I20260620 20:12:40.834328 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:40.833897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.129:58845)
0620 20:12:40.834177 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:40.834233 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":75,"threads_started":1}
I20260620 20:12:41.011793 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:41.011141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46243 (local address 127.5.91.130:35451)
0620 20:12:41.011543 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:41.011641 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.190:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":86,"threads_started":1}
I20260620 20:12:41.214391 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 22033
I20260620 20:12:41.219691 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 22207
I20260620 20:12:41.224496 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 22342
2026-06-20T20:12:41Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59203 ms)
[----------] 30 tests from SecurityITest (250078 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:12:41 dist-test-slave-l8h4 krb5kdc[22530](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:12:41 dist-test-slave-l8h4 krb5kdc[22530](info): set up 2 sockets
Jun 20 20:12:41 dist-test-slave-l8h4 krb5kdc[22530](info): commencing operation
krb5kdc: starting...
W20260620 20:12:42.202584 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:12:42 dist-test-slave-l8h4 krb5kdc[22530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:12:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:12:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:12:42.320959 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:41221
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:41221
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:12:42.431267 22543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:42.431531 22543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:42.431576 22543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:42.435773 22543 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:12:42.435837 22543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:42.435856 22543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:42.435871 22543 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:12:42.435885 22543 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:12:42.441114 22543 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:41221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:41221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:42.442067 22543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:42.443017 22543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:42.451263 22549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:42.451327 22543 server_base.cc:1034] running on GCE node
W20260620 20:12:42.451274 22552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:42.451274 22550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:42.452011 22543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:42.453058 22543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:42.454201 22543 hybrid_clock.cc:648] HybridClock initialized: now 1781986362454190 us; error 24 us; skew 500 ppm
Jun 20 20:12:42 dist-test-slave-l8h4 krb5kdc[22530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986362, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:42.456876 22543 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:12:42.457722 22543 webserver.cc:460] Webserver started at http://127.5.91.190:35569/ using document root <none> and password file <none>
I20260620 20:12:42.458279 22543 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:42.458348 22543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:42.458544 22543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:42.460467 22543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "30ee0fb76ae74218928d03ebd207a10c"
format_stamp: "Formatted at 2026-06-20 20:12:42 on dist-test-slave-l8h4"
server_key: "df9b48e325be0b63b31cbf721e48cec3"
server_key_iv: "801326b08cb87b652698e902db32663d"
server_key_version: "encryptionkey@0"
I20260620 20:12:42.460999 22543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "30ee0fb76ae74218928d03ebd207a10c"
format_stamp: "Formatted at 2026-06-20 20:12:42 on dist-test-slave-l8h4"
server_key: "df9b48e325be0b63b31cbf721e48cec3"
server_key_iv: "801326b08cb87b652698e902db32663d"
server_key_version: "encryptionkey@0"
I20260620 20:12:42.464573 22543 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 20:12:42.466885 22559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:42.468046 22543 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:12:42.468159 22543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "30ee0fb76ae74218928d03ebd207a10c"
format_stamp: "Formatted at 2026-06-20 20:12:42 on dist-test-slave-l8h4"
server_key: "df9b48e325be0b63b31cbf721e48cec3"
server_key_iv: "801326b08cb87b652698e902db32663d"
server_key_version: "encryptionkey@0"
I20260620 20:12:42.468250 22543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:42.510402 22543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:42.513425 22543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:42.513610 22543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:42.520116 22543 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:41221
I20260620 20:12:42.520138 22611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:41221 every 8 connection(s)
I20260620 20:12:42.521018 22543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:12:42.523751 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:42.527313 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 22543
I20260620 20:12:42.527441 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:12:42.527670 5486 external_mini_cluster.cc:1428] Setting key f5b162c90f942149993695583462e4e9
I20260620 20:12:42.529284 22612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c: Bootstrap starting.
I20260620 20:12:42.531879 22612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:42.532732 22612 log.cc:824] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:42.534787 22612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c: No bootstrap required, opened a new log
Jun 20 20:12:42 dist-test-slave-l8h4 krb5kdc[22530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:12:42.538955 22612 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30ee0fb76ae74218928d03ebd207a10c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41221 } }
I20260620 20:12:42.539234 22612 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:42.539289 22612 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ee0fb76ae74218928d03ebd207a10c, State: Initialized, Role: FOLLOWER
I20260620 20:12:42.539790 22612 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30ee0fb76ae74218928d03ebd207a10c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41221 } }
I20260620 20:12:42.539921 22612 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:12:42.539981 22612 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:12:42.540074 22612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:42.540961 22612 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30ee0fb76ae74218928d03ebd207a10c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41221 } }
I20260620 20:12:42.541280 22612 leader_election.cc:302] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30ee0fb76ae74218928d03ebd207a10c; no voters:
I20260620 20:12:42.541630 22612 leader_election.cc:288] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:12:42.541687 22617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:12:42.541915 22617 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c [term 1 LEADER]: Becoming Leader. State: Replica: 30ee0fb76ae74218928d03ebd207a10c, State: Running, Role: LEADER
I20260620 20:12:42.542302 22617 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30ee0fb76ae74218928d03ebd207a10c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41221 } }
I20260620 20:12:42.542712 22615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:42.528995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46828 (local address 127.5.91.190:41221)
0620 20:12:42.529429 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:42.529447 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:42.529488 (+ 41us) server_negotiation.cc:407] Connection header received
0620 20:12:42.530312 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:42.530333 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:42.530743 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:42.531091 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:42.531720 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:42.533173 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:42.534165 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:42.534842 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:42.534995 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:42.537554 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:42.537561 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:42.537579 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:42.537612 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:42.539612 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:42.540151 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:42.540157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:42.540163 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:42.540285 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:42.540540 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:42.540543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:42.540545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:42.540938 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:42.541112 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:42.541388 (+ 276us) server_negotiation.cc:299] Negotiation successful
0620 20:12:42.541706 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":110,"threads_started":1}
I20260620 20:12:42.542860 22612 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:12:42.543469 22617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30ee0fb76ae74218928d03ebd207a10c. Latest consensus state: current_term: 1 leader_uuid: "30ee0fb76ae74218928d03ebd207a10c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30ee0fb76ae74218928d03ebd207a10c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41221 } } }
I20260620 20:12:42.543581 22617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:42.543840 22618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30ee0fb76ae74218928d03ebd207a10c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30ee0fb76ae74218928d03ebd207a10c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41221 } } }
I20260620 20:12:42.543941 22618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c [sys.catalog]: This master's current role is: LEADER
W20260620 20:12:42.548398 22633 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 20:12:42.548473 22633 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 20:12:42.548554 22622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:12:42.549263 22622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:12:42.555090 22622 catalog_manager.cc:1347] Generated new cluster ID: fa75c4fd40bb47beb73ede23c8f1cd5e
I20260620 20:12:42.555158 22622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:12:42.562191 22622 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:12:42.563133 22622 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:12:42.572775 22622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c: Generated new TSK 0
I20260620 20:12:42.573606 22622 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.91.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
I20260620 20:12:42.599288 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41221
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:12:42.709057 22638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:42.709306 22638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:42.709347 22638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:42.713328 22638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:42.713385 22638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:42.713442 22638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:12:42.718667 22638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:42.719652 22638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:42.720650 22638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:42.729616 22643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:42.729701 22644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:42.729624 22646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:42.729883 22638 server_base.cc:1034] running on GCE node
I20260620 20:12:42.730367 22638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:42.730955 22638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:42.732148 22638 hybrid_clock.cc:648] HybridClock initialized: now 1781986362732124 us; error 37 us; skew 500 ppm
Jun 20 20:12:42 dist-test-slave-l8h4 krb5kdc[22530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986362, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:42.734864 22638 init.cc:377] Logged in from keytab as kudu/127.5.91.129@KRBTEST.COM (short username kudu)
I20260620 20:12:42.735679 22638 webserver.cc:460] Webserver started at http://127.5.91.129:34503/ using document root <none> and password file <none>
I20260620 20:12:42.736204 22638 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:42.736268 22638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:42.736442 22638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:42.738399 22638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "31cf822c3a1d4fdb875c2824e3514a07"
format_stamp: "Formatted at 2026-06-20 20:12:42 on dist-test-slave-l8h4"
server_key: "4ecb5ca09f0d227c345ef80dd7180bc9"
server_key_iv: "0aa6c20642b5d3842a7c877d621a7f1e"
server_key_version: "encryptionkey@0"
I20260620 20:12:42.738915 22638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "31cf822c3a1d4fdb875c2824e3514a07"
format_stamp: "Formatted at 2026-06-20 20:12:42 on dist-test-slave-l8h4"
server_key: "4ecb5ca09f0d227c345ef80dd7180bc9"
server_key_iv: "0aa6c20642b5d3842a7c877d621a7f1e"
server_key_version: "encryptionkey@0"
I20260620 20:12:42.742367 22638 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:12:42.744827 22653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:42.746244 22638 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:12:42.746337 22638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "31cf822c3a1d4fdb875c2824e3514a07"
format_stamp: "Formatted at 2026-06-20 20:12:42 on dist-test-slave-l8h4"
server_key: "4ecb5ca09f0d227c345ef80dd7180bc9"
server_key_iv: "0aa6c20642b5d3842a7c877d621a7f1e"
server_key_version: "encryptionkey@0"
I20260620 20:12:42.746420 22638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:42.779645 22638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:42.782753 22638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:42.782922 22638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:42.783474 22638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:42.784245 22638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:12:42.784289 22638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:42.784330 22638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:12:42.784359 22638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:42.792230 22638 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:45787
I20260620 20:12:42.792268 22767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:45787 every 8 connection(s)
I20260620 20:12:42.793061 22638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:12:42.795413 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 22638
I20260620 20:12:42.795557 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:12:42.795872 5486 external_mini_cluster.cc:1428] Setting key 64e1768ab52708561e74d227fd3221e3
Jun 20 20:12:42 dist-test-slave-l8h4 krb5kdc[22530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986362, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.91.130@KRBTEST.COM; defaulting to no policy
I20260620 20:12:42.807368 22615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:42.794743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:58987 (local address 127.5.91.190:41221)
0620 20:12:42.794869 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:42.794873 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:42.795742 (+ 869us) server_negotiation.cc:407] Connection header received
0620 20:12:42.797127 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:42.797130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:42.797180 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:42.797269 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:42.798710 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:42.799457 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:42.800914 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:42.801141 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:42.801221 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:42.803859 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:42.803861 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:42.803863 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:42.803887 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:42.805336 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:42.805820 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:42.805823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:42.805825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:42.805899 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:42.806260 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:42.806264 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:42.806265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:42.806441 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:42.806534 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:42.807047 (+ 513us) server_negotiation.cc:299] Negotiation successful
0620 20:12:42.807205 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.130@KRBTEST.COM" created.
I20260620 20:12:42.808152 22770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:42.795103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41221 (local address 127.5.91.129:58987)
0620 20:12:42.795583 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:42.795638 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:42.796914 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:42.797441 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:42.797450 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:42.797879 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:42.798573 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:42.798590 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:42.799622 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:42.799625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:42.800797 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:42.800808 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:42.801281 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:42.801285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:42.801493 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:42.802065 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:12:42.802084 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:42.803706 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:42.805438 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:42.805444 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:42.805457 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:42.805726 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:42.805979 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:42.805982 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:42.805984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:42.806179 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:42.806540 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:42.806547 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:42.806777 (+ 230us) client_negotiation.cc:769] Sending connection context
0620 20:12:42.806931 (+ 154us) client_negotiation.cc:240] Negotiation successful
0620 20:12:42.807239 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":100,"threads_started":1}
I20260620 20:12:42.809301 22768 heartbeater.cc:347] Connected to a master server at 127.5.91.190:41221
I20260620 20:12:42.809520 22768 heartbeater.cc:464] Registering TS with master...
I20260620 20:12:42.809978 22768 heartbeater.cc:511] Master 127.5.91.190:41221 requested a full tablet report, sending...
I20260620 20:12:42.811553 22576 ts_manager.cc:194] Registered new tserver with Master: 31cf822c3a1d4fdb875c2824e3514a07 (127.5.91.129:45787)
I20260620 20:12:42.813138 22576 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:58987
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
I20260620 20:12:42.823190 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41221
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:12:42.930904 22773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:42.931159 22773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:42.931206 22773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:42.935273 22773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:42.935339 22773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:42.935413 22773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:12:42.940636 22773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:42.941618 22773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:42.942627 22773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:42.952049 22778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:42.952049 22779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:42.952155 22781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:42.952224 22773 server_base.cc:1034] running on GCE node
I20260620 20:12:42.952754 22773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:42.953352 22773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:42.954531 22773 hybrid_clock.cc:648] HybridClock initialized: now 1781986362954507 us; error 38 us; skew 500 ppm
Jun 20 20:12:42 dist-test-slave-l8h4 krb5kdc[22530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986362, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:42.957158 22773 init.cc:377] Logged in from keytab as kudu/127.5.91.130@KRBTEST.COM (short username kudu)
I20260620 20:12:42.957983 22773 webserver.cc:460] Webserver started at http://127.5.91.130:33127/ using document root <none> and password file <none>
I20260620 20:12:42.958570 22773 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:42.958637 22773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:42.958813 22773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:42.960741 22773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "9feabbc8635048beb7712a4fd0b23798"
format_stamp: "Formatted at 2026-06-20 20:12:42 on dist-test-slave-l8h4"
server_key: "6501d1368523b2b3cc707e15494abdd0"
server_key_iv: "5181a9d108e908e5a030d65c998c064d"
server_key_version: "encryptionkey@0"
I20260620 20:12:42.961259 22773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "9feabbc8635048beb7712a4fd0b23798"
format_stamp: "Formatted at 2026-06-20 20:12:42 on dist-test-slave-l8h4"
server_key: "6501d1368523b2b3cc707e15494abdd0"
server_key_iv: "5181a9d108e908e5a030d65c998c064d"
server_key_version: "encryptionkey@0"
I20260620 20:12:42.964813 22773 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 20:12:42.967635 22788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:42.969370 22773 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 20:12:42.969570 22773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "9feabbc8635048beb7712a4fd0b23798"
format_stamp: "Formatted at 2026-06-20 20:12:42 on dist-test-slave-l8h4"
server_key: "6501d1368523b2b3cc707e15494abdd0"
server_key_iv: "5181a9d108e908e5a030d65c998c064d"
server_key_version: "encryptionkey@0"
I20260620 20:12:42.969719 22773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:42.991678 22773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:42.995136 22773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:42.995319 22773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:42.995890 22773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:42.996699 22773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:12:42.996747 22773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:42.996798 22773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:12:42.996824 22773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:43.005334 22773 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:34045
I20260620 20:12:43.005453 22902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:34045 every 8 connection(s)
I20260620 20:12:43.006418 22773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:12:43.009449 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 22773
I20260620 20:12:43.009579 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:12:43.009852 5486 external_mini_cluster.cc:1428] Setting key 4f2bfb1caf099899e65a543f636097fa
Jun 20 20:12:43 dist-test-slave-l8h4 krb5kdc[22530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986362, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.91.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.131@KRBTEST.COM" created.
I20260620 20:12:43.023936 22615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:43.008685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:54751 (local address 127.5.91.190:41221)
0620 20:12:43.008815 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:43.008818 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:43.010033 (+ 1215us) server_negotiation.cc:407] Connection header received
0620 20:12:43.011920 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:43.011923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:43.011978 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:43.012114 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:43.013956 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.014646 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.016696 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.016960 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.017103 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:43.020215 (+ 3112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:43.020219 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:43.020221 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:43.020248 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:43.021622 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:43.022125 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:43.022128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:43.022130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:43.022242 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:43.022743 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:43.022746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:43.022747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:43.022949 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:43.023063 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:43.023687 (+ 624us) server_negotiation.cc:299] Negotiation successful
0620 20:12:43.023806 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 20:12:43.024978 22905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:43.009331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41221 (local address 127.5.91.130:54751)
0620 20:12:43.009848 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:43.009900 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:43.011644 (+ 1744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:43.012302 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:43.012310 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:43.012801 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:43.013702 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:43.013727 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.014813 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.014817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:43.016491 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:43.016516 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.017199 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.017204 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:43.017432 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:43.018105 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:12:43.018129 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:43.019980 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:43.021725 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:43.021730 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:43.021743 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:43.021979 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:43.022321 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:43.022325 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:43.022327 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:43.022469 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:43.023090 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:43.023097 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:43.023395 (+ 298us) client_negotiation.cc:769] Sending connection context
0620 20:12:43.023616 (+ 221us) client_negotiation.cc:240] Negotiation successful
0620 20:12:43.023976 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":116,"threads_started":1}
I20260620 20:12:43.026160 22903 heartbeater.cc:347] Connected to a master server at 127.5.91.190:41221
I20260620 20:12:43.026388 22903 heartbeater.cc:464] Registering TS with master...
I20260620 20:12:43.026868 22903 heartbeater.cc:511] Master 127.5.91.190:41221 requested a full tablet report, sending...
I20260620 20:12:43.027949 22576 ts_manager.cc:194] Registered new tserver with Master: 9feabbc8635048beb7712a4fd0b23798 (127.5.91.130:34045)
I20260620 20:12:43.028513 22576 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:54751
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
I20260620 20:12:43.039261 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41221
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:12:43.144766 22908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:43.145123 22908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:43.145197 22908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:43.149549 22908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:43.149609 22908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:43.149685 22908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:12:43.154979 22908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:43.155999 22908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:43.156960 22908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:43.166682 22916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:43.166685 22914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:43.166682 22913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:43.166886 22908 server_base.cc:1034] running on GCE node
I20260620 20:12:43.167333 22908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:43.167949 22908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:43.169122 22908 hybrid_clock.cc:648] HybridClock initialized: now 1781986363169099 us; error 35 us; skew 500 ppm
Jun 20 20:12:43 dist-test-slave-l8h4 krb5kdc[22530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986363, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:43.172168 22908 init.cc:377] Logged in from keytab as kudu/127.5.91.131@KRBTEST.COM (short username kudu)
I20260620 20:12:43.173189 22908 webserver.cc:460] Webserver started at http://127.5.91.131:36959/ using document root <none> and password file <none>
I20260620 20:12:43.173748 22908 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:43.173812 22908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:43.174045 22908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:43.176240 22908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "f986b7bb7cbc4ed089409517d4263c0b"
format_stamp: "Formatted at 2026-06-20 20:12:43 on dist-test-slave-l8h4"
server_key: "63e8a41dfe12adaee56c955e07c88475"
server_key_iv: "2f9680e454fe5d5ae4e09b9ab9e0b92a"
server_key_version: "encryptionkey@0"
I20260620 20:12:43.177178 22908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "f986b7bb7cbc4ed089409517d4263c0b"
format_stamp: "Formatted at 2026-06-20 20:12:43 on dist-test-slave-l8h4"
server_key: "63e8a41dfe12adaee56c955e07c88475"
server_key_iv: "2f9680e454fe5d5ae4e09b9ab9e0b92a"
server_key_version: "encryptionkey@0"
I20260620 20:12:43.181437 22908 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 20:12:43.183874 22923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:43.185734 22908 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 20:12:43.186067 22908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "f986b7bb7cbc4ed089409517d4263c0b"
format_stamp: "Formatted at 2026-06-20 20:12:43 on dist-test-slave-l8h4"
server_key: "63e8a41dfe12adaee56c955e07c88475"
server_key_iv: "2f9680e454fe5d5ae4e09b9ab9e0b92a"
server_key_version: "encryptionkey@0"
I20260620 20:12:43.186276 22908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:43.206408 22908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:43.209400 22908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:43.209579 22908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:43.210301 22908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:43.211599 22908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:12:43.211738 22908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:43.211846 22908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:12:43.211884 22908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:43.222664 22908 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:46675
I20260620 20:12:43.222684 23037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:46675 every 8 connection(s)
I20260620 20:12:43.224032 22908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:12:43.225488 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 22908
I20260620 20:12:43.225626 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:12:43.226120 5486 external_mini_cluster.cc:1428] Setting key 49c28e37d4388784cf46bf742de2ae5f
Jun 20 20:12:43 dist-test-slave-l8h4 krb5kdc[22530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986363, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:12:43.241534 22615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:43.226534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:42303 (local address 127.5.91.190:41221)
0620 20:12:43.226674 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:43.226678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:43.227958 (+ 1280us) server_negotiation.cc:407] Connection header received
0620 20:12:43.229637 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:43.229641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:43.229696 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:43.229767 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:43.232069 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.232655 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.233845 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.234074 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.234205 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:43.236796 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:43.236799 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:43.236801 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:43.236827 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:43.238554 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:43.239149 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:43.239152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:43.239153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:43.239250 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:43.239962 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:43.239966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:43.239967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:43.240360 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:43.240522 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:43.241220 (+ 698us) server_negotiation.cc:299] Negotiation successful
0620 20:12:43.241402 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 20:12:43.242403 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:43.227065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41221 (local address 127.5.91.131:42303)
0620 20:12:43.227804 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:43.227850 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:43.229316 (+ 1466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:43.230080 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:43.230101 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:43.230988 (+ 887us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:43.231823 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:43.231847 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.232791 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.232794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:43.233716 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:43.233727 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.234250 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.234254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:43.234432 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:43.235023 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:12:43.235042 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:43.236615 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:43.238685 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:43.238693 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:43.238707 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:43.239015 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:43.239376 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:43.239379 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:43.239380 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:43.239570 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:43.240542 (+ 972us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:43.240549 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:43.240895 (+ 346us) client_negotiation.cc:769] Sending connection context
0620 20:12:43.241128 (+ 233us) client_negotiation.cc:240] Negotiation successful
0620 20:12:43.241453 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":138,"threads_started":1}
I20260620 20:12:43.243546 23038 heartbeater.cc:347] Connected to a master server at 127.5.91.190:41221
I20260620 20:12:43.243762 23038 heartbeater.cc:464] Registering TS with master...
I20260620 20:12:43.244302 23038 heartbeater.cc:511] Master 127.5.91.190:41221 requested a full tablet report, sending...
I20260620 20:12:43.245334 22576 ts_manager.cc:194] Registered new tserver with Master: f986b7bb7cbc4ed089409517d4263c0b (127.5.91.131:46675)
I20260620 20:12:43.245929 22576 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:42303
I20260620 20:12:43.253199 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 20:12:43.262228 22615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:43.255385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47104 (local address 127.5.91.190:41221)
0620 20:12:43.255529 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:43.255533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:43.255643 (+ 110us) server_negotiation.cc:407] Connection header received
0620 20:12:43.255763 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:43.255766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:43.255812 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:43.255875 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:43.256557 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.257080 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.257902 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.258117 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.258237 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:43.259500 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:43.259504 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:43.259507 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:43.259530 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:43.260687 (+ 1157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:43.261058 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:43.261062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:43.261064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:43.261139 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:43.261369 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:43.261372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:43.261374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:43.261605 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:43.261726 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:43.261937 (+ 211us) server_negotiation.cc:299] Negotiation successful
0620 20:12:43.262046 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 20:12:43.265311 22576 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47104:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:12:43.268493 22576 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:12:43.283283 23051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:43.277494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53948 (local address 127.5.91.129:45787)
0620 20:12:43.277825 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:43.277843 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:43.278073 (+ 230us) server_negotiation.cc:407] Connection header received
0620 20:12:43.278411 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:43.278416 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:43.278631 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:43.278753 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:43.279413 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.280266 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.281733 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.282615 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.282686 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:43.282835 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:43.283034 (+ 199us) server_negotiation.cc:299] Negotiation successful
0620 20:12:43.283141 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":76,"threads_started":1}
I20260620 20:12:43.283275 23049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:43.277355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:45787 (local address 127.0.0.1:53948)
0620 20:12:43.277932 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:43.277984 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:43.278219 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:43.278965 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:43.278971 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:43.278991 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:43.279261 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:43.279273 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.280390 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.280392 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:43.281586 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:43.281596 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.282756 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.282758 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:43.282858 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:43.282895 (+ 37us) client_negotiation.cc:769] Sending connection context
0620 20:12:43.282989 (+ 94us) client_negotiation.cc:240] Negotiation successful
0620 20:12:43.283119 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":100,"thread_start_us":84,"threads_started":1}
I20260620 20:12:43.283618 23050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:43.277431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:34045 (local address 127.0.0.1:53564)
0620 20:12:43.278332 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:43.278344 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:43.278417 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:43.279037 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:43.279041 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:43.279058 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:43.279267 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:43.279275 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.280927 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.280930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:43.282216 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:43.282226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.283236 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.283240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:43.283344 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:43.283369 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 20:12:43.283417 (+ 48us) client_negotiation.cc:240] Negotiation successful
0620 20:12:43.283483 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"mutex_wait_us":103,"thread_start_us":77,"threads_started":1}
I20260620 20:12:43.284104 23052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:43.278043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:46675 (local address 127.0.0.1:38718)
0620 20:12:43.278593 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:43.278605 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:43.278671 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:43.279424 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:43.279426 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:43.279445 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:43.279535 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:43.279541 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.281233 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.281236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:43.282590 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:43.282601 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.283720 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.283724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:43.283819 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:43.283842 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 20:12:43.283917 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 20:12:43.283987 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":172,"thread_start_us":105,"threads_started":1}
I20260620 20:12:43.284174 23054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:43.278275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38718 (local address 127.5.91.131:46675)
0620 20:12:43.278826 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:43.278838 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:43.278857 (+ 19us) server_negotiation.cc:407] Connection header received
0620 20:12:43.278924 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:43.278928 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:43.279159 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:43.279318 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:43.279676 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.280791 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.282713 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.283604 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.283710 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:43.283841 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:43.283952 (+ 111us) server_negotiation.cc:299] Negotiation successful
0620 20:12:43.284042 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":140,"threads_started":1}
I20260620 20:12:43.284282 23053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:43.278082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53564 (local address 127.5.91.130:34045)
0620 20:12:43.278477 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:43.278488 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:43.278509 (+ 21us) server_negotiation.cc:407] Connection header received
0620 20:12:43.278581 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:43.278585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:43.278893 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:43.279402 (+ 509us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:43.279570 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.280791 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.282349 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.283020 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.283889 (+ 869us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:43.284026 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:43.284069 (+ 43us) server_negotiation.cc:299] Negotiation successful
0620 20:12:43.284152 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":127,"threads_started":1}
I20260620 20:12:43.286700 22836 tablet_service.cc:1467] Processing CreateTablet for tablet b8023d9550894f489265f4af1155d911 (DEFAULT_TABLE table=test-table [id=a44132016a0242d7b22de29c3ef3951d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:12:43.286722 22702 tablet_service.cc:1467] Processing CreateTablet for tablet b8023d9550894f489265f4af1155d911 (DEFAULT_TABLE table=test-table [id=a44132016a0242d7b22de29c3ef3951d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:12:43.286722 22972 tablet_service.cc:1467] Processing CreateTablet for tablet b8023d9550894f489265f4af1155d911 (DEFAULT_TABLE table=test-table [id=a44132016a0242d7b22de29c3ef3951d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:12:43.287671 22836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8023d9550894f489265f4af1155d911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:43.287700 22702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8023d9550894f489265f4af1155d911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:43.287705 22972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8023d9550894f489265f4af1155d911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:43.293817 23055 tablet_bootstrap.cc:492] T b8023d9550894f489265f4af1155d911 P 9feabbc8635048beb7712a4fd0b23798: Bootstrap starting.
I20260620 20:12:43.295303 23057 tablet_bootstrap.cc:492] T b8023d9550894f489265f4af1155d911 P 31cf822c3a1d4fdb875c2824e3514a07: Bootstrap starting.
I20260620 20:12:43.296216 23055 tablet_bootstrap.cc:654] T b8023d9550894f489265f4af1155d911 P 9feabbc8635048beb7712a4fd0b23798: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:43.297183 23055 log.cc:824] T b8023d9550894f489265f4af1155d911 P 9feabbc8635048beb7712a4fd0b23798: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:43.297318 23057 tablet_bootstrap.cc:654] T b8023d9550894f489265f4af1155d911 P 31cf822c3a1d4fdb875c2824e3514a07: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:43.297976 23056 tablet_bootstrap.cc:492] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b: Bootstrap starting.
I20260620 20:12:43.298460 23057 log.cc:824] T b8023d9550894f489265f4af1155d911 P 31cf822c3a1d4fdb875c2824e3514a07: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:43.299378 23055 tablet_bootstrap.cc:492] T b8023d9550894f489265f4af1155d911 P 9feabbc8635048beb7712a4fd0b23798: No bootstrap required, opened a new log
I20260620 20:12:43.299582 23055 ts_tablet_manager.cc:1397] T b8023d9550894f489265f4af1155d911 P 9feabbc8635048beb7712a4fd0b23798: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 20:12:43.300024 23057 tablet_bootstrap.cc:492] T b8023d9550894f489265f4af1155d911 P 31cf822c3a1d4fdb875c2824e3514a07: No bootstrap required, opened a new log
I20260620 20:12:43.300196 23057 ts_tablet_manager.cc:1397] T b8023d9550894f489265f4af1155d911 P 31cf822c3a1d4fdb875c2824e3514a07: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 20:12:43.300273 23056 tablet_bootstrap.cc:654] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:43.301448 23056 log.cc:824] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:43.303298 23055 raft_consensus.cc:348] T b8023d9550894f489265f4af1155d911 P 9feabbc8635048beb7712a4fd0b23798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cf822c3a1d4fdb875c2824e3514a07" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 45787 } } peers { permanent_uuid: "9feabbc8635048beb7712a4fd0b23798" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 34045 } } peers { permanent_uuid: "f986b7bb7cbc4ed089409517d4263c0b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46675 } }
I20260620 20:12:43.303465 23056 tablet_bootstrap.cc:492] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b: No bootstrap required, opened a new log
I20260620 20:12:43.303522 23055 raft_consensus.cc:374] T b8023d9550894f489265f4af1155d911 P 9feabbc8635048beb7712a4fd0b23798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:43.303599 23055 raft_consensus.cc:729] T b8023d9550894f489265f4af1155d911 P 9feabbc8635048beb7712a4fd0b23798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9feabbc8635048beb7712a4fd0b23798, State: Initialized, Role: FOLLOWER
I20260620 20:12:43.303646 23056 ts_tablet_manager.cc:1397] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 20:12:43.303625 23057 raft_consensus.cc:348] T b8023d9550894f489265f4af1155d911 P 31cf822c3a1d4fdb875c2824e3514a07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cf822c3a1d4fdb875c2824e3514a07" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 45787 } } peers { permanent_uuid: "9feabbc8635048beb7712a4fd0b23798" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 34045 } } peers { permanent_uuid: "f986b7bb7cbc4ed089409517d4263c0b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46675 } }
I20260620 20:12:43.303853 23057 raft_consensus.cc:374] T b8023d9550894f489265f4af1155d911 P 31cf822c3a1d4fdb875c2824e3514a07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:43.303941 23057 raft_consensus.cc:729] T b8023d9550894f489265f4af1155d911 P 31cf822c3a1d4fdb875c2824e3514a07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31cf822c3a1d4fdb875c2824e3514a07, State: Initialized, Role: FOLLOWER
I20260620 20:12:43.304046 23055 consensus_queue.cc:260] T b8023d9550894f489265f4af1155d911 P 9feabbc8635048beb7712a4fd0b23798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cf822c3a1d4fdb875c2824e3514a07" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 45787 } } peers { permanent_uuid: "9feabbc8635048beb7712a4fd0b23798" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 34045 } } peers { permanent_uuid: "f986b7bb7cbc4ed089409517d4263c0b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46675 } }
I20260620 20:12:43.304352 23057 consensus_queue.cc:260] T b8023d9550894f489265f4af1155d911 P 31cf822c3a1d4fdb875c2824e3514a07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cf822c3a1d4fdb875c2824e3514a07" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 45787 } } peers { permanent_uuid: "9feabbc8635048beb7712a4fd0b23798" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 34045 } } peers { permanent_uuid: "f986b7bb7cbc4ed089409517d4263c0b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46675 } }
I20260620 20:12:43.304670 22903 heartbeater.cc:503] Master 127.5.91.190:41221 was elected leader, sending a full tablet report...
I20260620 20:12:43.305017 23055 ts_tablet_manager.cc:1428] T b8023d9550894f489265f4af1155d911 P 9feabbc8635048beb7712a4fd0b23798: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260620 20:12:43.305285 23057 ts_tablet_manager.cc:1428] T b8023d9550894f489265f4af1155d911 P 31cf822c3a1d4fdb875c2824e3514a07: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260620 20:12:43.305605 22768 heartbeater.cc:503] Master 127.5.91.190:41221 was elected leader, sending a full tablet report...
I20260620 20:12:43.307183 23056 raft_consensus.cc:348] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cf822c3a1d4fdb875c2824e3514a07" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 45787 } } peers { permanent_uuid: "9feabbc8635048beb7712a4fd0b23798" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 34045 } } peers { permanent_uuid: "f986b7bb7cbc4ed089409517d4263c0b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46675 } }
I20260620 20:12:43.307441 23056 raft_consensus.cc:374] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:43.307536 23056 raft_consensus.cc:729] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f986b7bb7cbc4ed089409517d4263c0b, State: Initialized, Role: FOLLOWER
I20260620 20:12:43.308241 23056 consensus_queue.cc:260] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cf822c3a1d4fdb875c2824e3514a07" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 45787 } } peers { permanent_uuid: "9feabbc8635048beb7712a4fd0b23798" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 34045 } } peers { permanent_uuid: "f986b7bb7cbc4ed089409517d4263c0b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46675 } }
I20260620 20:12:43.309230 23056 ts_tablet_manager.cc:1428] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260620 20:12:43.309518 23038 heartbeater.cc:503] Master 127.5.91.190:41221 was elected leader, sending a full tablet report...
I20260620 20:12:43.446626 23063 raft_consensus.cc:482] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:12:43.446841 23063 raft_consensus.cc:504] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cf822c3a1d4fdb875c2824e3514a07" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 45787 } } peers { permanent_uuid: "9feabbc8635048beb7712a4fd0b23798" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 34045 } } peers { permanent_uuid: "f986b7bb7cbc4ed089409517d4263c0b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46675 } }
I20260620 20:12:43.447945 23063 leader_election.cc:288] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31cf822c3a1d4fdb875c2824e3514a07 (127.5.91.129:45787), 9feabbc8635048beb7712a4fd0b23798 (127.5.91.130:34045)
I20260620 20:12:43.452314 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:43.448219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:45787 (local address 127.5.91.131:33721)
0620 20:12:43.448369 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:43.448382 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:43.448464 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:43.448907 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:43.448910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:43.448923 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:43.449033 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:43.449040 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.450189 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.450192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:43.451174 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:43.451184 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.451912 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.451916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:43.452028 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:43.452054 (+ 26us) client_negotiation.cc:769] Sending connection context
0620 20:12:43.452123 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 20:12:43.452199 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":113}
I20260620 20:12:43.452343 23051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:43.448570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:33721 (local address 127.5.91.129:45787)
0620 20:12:43.448712 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:43.448717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:43.448731 (+ 14us) server_negotiation.cc:407] Connection header received
0620 20:12:43.448785 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:43.448788 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:43.448834 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:43.448901 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:43.449139 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.449998 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.451307 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.451793 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.451885 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:43.451926 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:43.452145 (+ 219us) server_negotiation.cc:299] Negotiation successful
0620 20:12:43.452220 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 20:12:43.452499 23064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:43.448308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:34045 (local address 127.5.91.131:48877)
0620 20:12:43.448673 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:43.448684 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:43.448789 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:43.449049 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:43.449051 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:43.449066 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:43.449169 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:43.449174 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.450197 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.450201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:43.451379 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:43.451390 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.452100 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.452104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:43.452204 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:43.452233 (+ 29us) client_negotiation.cc:769] Sending connection context
0620 20:12:43.452304 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 20:12:43.452365 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":23,"thread_start_us":127,"threads_started":1}
I20260620 20:12:43.452526 23053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:43.448399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:48877 (local address 127.5.91.130:34045)
0620 20:12:43.448532 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:43.448537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:43.448768 (+ 231us) server_negotiation.cc:407] Connection header received
0620 20:12:43.448881 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:43.448883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:43.448932 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:43.449033 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:43.449255 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.450042 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.451535 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.451986 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.452103 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:43.452148 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:43.452328 (+ 180us) server_negotiation.cc:299] Negotiation successful
0620 20:12:43.452400 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 20:12:43.453193 22722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8023d9550894f489265f4af1155d911" candidate_uuid: "f986b7bb7cbc4ed089409517d4263c0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31cf822c3a1d4fdb875c2824e3514a07" is_pre_election: true
I20260620 20:12:43.453604 22722 raft_consensus.cc:2454] T b8023d9550894f489265f4af1155d911 P 31cf822c3a1d4fdb875c2824e3514a07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f986b7bb7cbc4ed089409517d4263c0b in term 0.
I20260620 20:12:43.454200 22927 leader_election.cc:302] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31cf822c3a1d4fdb875c2824e3514a07, f986b7bb7cbc4ed089409517d4263c0b; no voters:
I20260620 20:12:43.454246 22857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8023d9550894f489265f4af1155d911" candidate_uuid: "f986b7bb7cbc4ed089409517d4263c0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9feabbc8635048beb7712a4fd0b23798" is_pre_election: true
I20260620 20:12:43.454499 23063 raft_consensus.cc:2781] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:12:43.454526 22857 raft_consensus.cc:2454] T b8023d9550894f489265f4af1155d911 P 9feabbc8635048beb7712a4fd0b23798 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f986b7bb7cbc4ed089409517d4263c0b in term 0.
I20260620 20:12:43.454576 23063 raft_consensus.cc:482] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:12:43.454607 23063 raft_consensus.cc:3037] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:43.455629 23063 raft_consensus.cc:504] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cf822c3a1d4fdb875c2824e3514a07" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 45787 } } peers { permanent_uuid: "9feabbc8635048beb7712a4fd0b23798" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 34045 } } peers { permanent_uuid: "f986b7bb7cbc4ed089409517d4263c0b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46675 } }
I20260620 20:12:43.455966 23063 leader_election.cc:288] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b [CANDIDATE]: Term 1 election: Requested vote from peers 31cf822c3a1d4fdb875c2824e3514a07 (127.5.91.129:45787), 9feabbc8635048beb7712a4fd0b23798 (127.5.91.130:34045)
I20260620 20:12:43.456342 22857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8023d9550894f489265f4af1155d911" candidate_uuid: "f986b7bb7cbc4ed089409517d4263c0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9feabbc8635048beb7712a4fd0b23798"
I20260620 20:12:43.456342 22722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8023d9550894f489265f4af1155d911" candidate_uuid: "f986b7bb7cbc4ed089409517d4263c0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31cf822c3a1d4fdb875c2824e3514a07"
I20260620 20:12:43.456478 22857 raft_consensus.cc:3037] T b8023d9550894f489265f4af1155d911 P 9feabbc8635048beb7712a4fd0b23798 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:43.456461 22722 raft_consensus.cc:3037] T b8023d9550894f489265f4af1155d911 P 31cf822c3a1d4fdb875c2824e3514a07 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:43.457700 22722 raft_consensus.cc:2454] T b8023d9550894f489265f4af1155d911 P 31cf822c3a1d4fdb875c2824e3514a07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f986b7bb7cbc4ed089409517d4263c0b in term 1.
I20260620 20:12:43.457700 22857 raft_consensus.cc:2454] T b8023d9550894f489265f4af1155d911 P 9feabbc8635048beb7712a4fd0b23798 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f986b7bb7cbc4ed089409517d4263c0b in term 1.
I20260620 20:12:43.458007 22925 leader_election.cc:302] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9feabbc8635048beb7712a4fd0b23798, f986b7bb7cbc4ed089409517d4263c0b; no voters:
I20260620 20:12:43.458181 23063 raft_consensus.cc:2781] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:12:43.458391 23063 raft_consensus.cc:686] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b [term 1 LEADER]: Becoming Leader. State: Replica: f986b7bb7cbc4ed089409517d4263c0b, State: Running, Role: LEADER
I20260620 20:12:43.458690 23063 consensus_queue.cc:237] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cf822c3a1d4fdb875c2824e3514a07" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 45787 } } peers { permanent_uuid: "9feabbc8635048beb7712a4fd0b23798" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 34045 } } peers { permanent_uuid: "f986b7bb7cbc4ed089409517d4263c0b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46675 } }
I20260620 20:12:43.461885 22576 catalog_manager.cc:5495] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b reported cstate change: term changed from 0 to 1, leader changed from <none> to f986b7bb7cbc4ed089409517d4263c0b (127.5.91.131). New cstate: current_term: 1 leader_uuid: "f986b7bb7cbc4ed089409517d4263c0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cf822c3a1d4fdb875c2824e3514a07" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 45787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9feabbc8635048beb7712a4fd0b23798" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 34045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f986b7bb7cbc4ed089409517d4263c0b" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46675 } health_report { overall_health: HEALTHY } } }
I20260620 20:12:43.474359 23054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:43.470540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38734 (local address 127.5.91.131:46675)
0620 20:12:43.470647 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:43.470651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:43.470663 (+ 12us) server_negotiation.cc:407] Connection header received
0620 20:12:43.470749 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:43.470751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:43.470802 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:43.470881 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:12:43.471154 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.471688 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.472738 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.472950 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.473031 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:43.473325 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:12:43.473856 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:12:43.473959 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:43.474039 (+ 80us) server_negotiation.cc:299] Negotiation successful
0620 20:12:43.474101 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260620 20:12:43.475308 23039 tablet.cc:2369] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:12:43.479897 22857 raft_consensus.cc:1261] T b8023d9550894f489265f4af1155d911 P 9feabbc8635048beb7712a4fd0b23798 [term 1 FOLLOWER]: Refusing update from remote peer f986b7bb7cbc4ed089409517d4263c0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:12:43.479923 22722 raft_consensus.cc:1261] T b8023d9550894f489265f4af1155d911 P 31cf822c3a1d4fdb875c2824e3514a07 [term 1 FOLLOWER]: Refusing update from remote peer f986b7bb7cbc4ed089409517d4263c0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:12:43.480557 23065 consensus_queue.cc:1035] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9feabbc8635048beb7712a4fd0b23798" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 34045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:12:43.480727 23063 consensus_queue.cc:1035] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "31cf822c3a1d4fdb875c2824e3514a07" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:12:43.491046 23069 mvcc.cc:205] Tried to move back new op lower bound from 7299016144807292928 to 7299016144729493504. Current Snapshot: MvccSnapshot[applied={T|T < 7299016144807292928}]
I20260620 20:12:43.496824 22574 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:12:43.496984 22574 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:12:43.499445 22574 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 30ee0fb76ae74218928d03ebd207a10c: Sending DeleteTablet for 3 replicas of tablet b8023d9550894f489265f4af1155d911
I20260620 20:12:43.500253 22836 tablet_service.cc:1514] Processing DeleteTablet for tablet b8023d9550894f489265f4af1155d911 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:12:43 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:53564
I20260620 20:12:43.500324 22972 tablet_service.cc:1514] Processing DeleteTablet for tablet b8023d9550894f489265f4af1155d911 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:12:43 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:38718
I20260620 20:12:43.500363 22702 tablet_service.cc:1514] Processing DeleteTablet for tablet b8023d9550894f489265f4af1155d911 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:12:43 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:53948
I20260620 20:12:43.500923 23081 tablet_replica.cc:330] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b: stopping tablet replica
I20260620 20:12:43.500941 23079 tablet_replica.cc:330] T b8023d9550894f489265f4af1155d911 P 9feabbc8635048beb7712a4fd0b23798: stopping tablet replica
I20260620 20:12:43.501250 23081 raft_consensus.cc:2229] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b [term 1 LEADER]: Raft consensus shutting down.
I20260620 20:12:43.501259 23079 raft_consensus.cc:2229] T b8023d9550894f489265f4af1155d911 P 9feabbc8635048beb7712a4fd0b23798 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:12:43.501475 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 22638
I20260620 20:12:43.501578 23079 raft_consensus.cc:2258] T b8023d9550894f489265f4af1155d911 P 9feabbc8635048beb7712a4fd0b23798 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:12:43.501727 23081 raft_consensus.cc:2258] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:12:43.503247 23079 ts_tablet_manager.cc:1905] T b8023d9550894f489265f4af1155d911 P 9feabbc8635048beb7712a4fd0b23798: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:12:43.503638 23081 ts_tablet_manager.cc:1905] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:12:43.506084 23079 ts_tablet_manager.cc:1918] T b8023d9550894f489265f4af1155d911 P 9feabbc8635048beb7712a4fd0b23798: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:12:43.506100 23081 ts_tablet_manager.cc:1918] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:12:43.506201 23081 log.cc:1196] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/b8023d9550894f489265f4af1155d911
I20260620 20:12:43.506203 23079 log.cc:1196] T b8023d9550894f489265f4af1155d911 P 9feabbc8635048beb7712a4fd0b23798: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781986111135528-5486-0/minicluster-data/ts-1/wal/wals/b8023d9550894f489265f4af1155d911
I20260620 20:12:43.506472 23081 ts_tablet_manager.cc:1939] T b8023d9550894f489265f4af1155d911 P f986b7bb7cbc4ed089409517d4263c0b: Deleting consensus metadata
I20260620 20:12:43.506489 23079 ts_tablet_manager.cc:1939] T b8023d9550894f489265f4af1155d911 P 9feabbc8635048beb7712a4fd0b23798: Deleting consensus metadata
I20260620 20:12:43.507544 22563 catalog_manager.cc:4841] TS f986b7bb7cbc4ed089409517d4263c0b (127.5.91.131:46675): tablet b8023d9550894f489265f4af1155d911 (table test-table [id=a44132016a0242d7b22de29c3ef3951d]) successfully deleted
I20260620 20:12:43.507596 22561 catalog_manager.cc:4841] TS 9feabbc8635048beb7712a4fd0b23798 (127.5.91.130:34045): tablet b8023d9550894f489265f4af1155d911 (table test-table [id=a44132016a0242d7b22de29c3ef3951d]) successfully deleted
W20260620 20:12:43.511044 22563 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.91.129:45787 (error 108)
I20260620 20:12:43.511097 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 22773
I20260620 20:12:43.514266 23052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:43.513869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:45787 (local address 127.0.0.1:53958)
0620 20:12:43.514024 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:43.514114 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.129:45787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260620 20:12:43.514526 22563 catalog_manager.cc:4584] TS 31cf822c3a1d4fdb875c2824e3514a07 (127.5.91.129:45787): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b8023d9550894f489265f4af1155d911: Network error: Client connection negotiation failed: client connection to 127.5.91.129:45787: connect: Connection refused (error 111)
I20260620 20:12:43.518797 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 22908
I20260620 20:12:43.523766 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 22543
2026-06-20T20:12:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2298 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-20T20:12:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:12:43Z Disabled control of system clock
I20260620 20:12:43.551246 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:40539
--webserver_interface=127.5.91.190
--webserver_port=0
--builtin_ntp_servers=127.5.91.148:33987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:40539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 20:12:43.658530 23086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:43.658797 23086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:43.658833 23086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:43.662794 23086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:12:43.662849 23086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:43.662863 23086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:43.662879 23086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:12:43.662894 23086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:12:43.667616 23086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:33987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:40539
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:40539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:43.668637 23086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:43.669906 23086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:43.678423 23094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:43.678566 23086 server_base.cc:1034] running on GCE node
W20260620 20:12:43.678423 23091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:43.678503 23092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:43.679423 23086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:43.680506 23086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:43.681774 23086 hybrid_clock.cc:648] HybridClock initialized: now 1781986363681731 us; error 54 us; skew 500 ppm
I20260620 20:12:43.683693 23086 webserver.cc:460] Webserver started at http://127.5.91.190:37149/ using document root <none> and password file <none>
I20260620 20:12:43.684567 23086 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:43.684713 23086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:43.685070 23086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:43.687187 23086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "18265af27667495a91630b5d11727776"
format_stamp: "Formatted at 2026-06-20 20:12:43 on dist-test-slave-l8h4"
server_key: "b2a812f33378c9c93bf54c702ac54ea2"
server_key_iv: "fc520450eeb45519fe2001da79c01a29"
server_key_version: "encryptionkey@0"
I20260620 20:12:43.687716 23086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "18265af27667495a91630b5d11727776"
format_stamp: "Formatted at 2026-06-20 20:12:43 on dist-test-slave-l8h4"
server_key: "b2a812f33378c9c93bf54c702ac54ea2"
server_key_iv: "fc520450eeb45519fe2001da79c01a29"
server_key_version: "encryptionkey@0"
I20260620 20:12:43.691310 23086 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 20:12:43.693640 23100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:43.695191 23086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260620 20:12:43.695333 23086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "18265af27667495a91630b5d11727776"
format_stamp: "Formatted at 2026-06-20 20:12:43 on dist-test-slave-l8h4"
server_key: "b2a812f33378c9c93bf54c702ac54ea2"
server_key_iv: "fc520450eeb45519fe2001da79c01a29"
server_key_version: "encryptionkey@0"
I20260620 20:12:43.695436 23086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:43.722702 23086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:43.723438 23086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:43.723805 23086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:43.731382 23086 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:40539
I20260620 20:12:43.731439 23152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:40539 every 8 connection(s)
I20260620 20:12:43.732352 23086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:12:43.735440 23153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:43.737044 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 23086
I20260620 20:12:43.737179 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:12:43.737448 5486 external_mini_cluster.cc:1428] Setting key 988238d91952e3e311df665a00ef6488
I20260620 20:12:43.742846 23153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776: Bootstrap starting.
I20260620 20:12:43.745107 23153 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:43.745797 23153 log.cc:824] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:43.746773 23156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:43.738918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48800 (local address 127.5.91.190:40539)
0620 20:12:43.739605 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:43.739621 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:43.739657 (+ 36us) server_negotiation.cc:407] Connection header received
0620 20:12:43.740339 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:43.740359 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:43.740788 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:43.741110 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:43.741537 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.742578 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.743431 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.743909 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.744049 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:43.744928 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:43.744933 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:43.744945 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:12:43.745358 (+ 413us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:43.745415 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:12:43.745435 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:43.745531 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:43.745657 (+ 126us) server_negotiation.cc:299] Negotiation successful
0620 20:12:43.745911 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":134,"threads_started":1}
I20260620 20:12:43.747808 23153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776: No bootstrap required, opened a new log
I20260620 20:12:43.751303 23153 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18265af27667495a91630b5d11727776" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40539 } }
I20260620 20:12:43.751516 23153 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:43.751598 23153 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18265af27667495a91630b5d11727776, State: Initialized, Role: FOLLOWER
I20260620 20:12:43.752025 23153 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18265af27667495a91630b5d11727776" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40539 } }
I20260620 20:12:43.752188 23153 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:12:43.752260 23153 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:12:43.752354 23153 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:43.753439 23153 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18265af27667495a91630b5d11727776" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40539 } }
I20260620 20:12:43.753742 23153 leader_election.cc:302] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18265af27667495a91630b5d11727776; no voters:
I20260620 20:12:43.754019 23153 leader_election.cc:288] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:12:43.754128 23158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:12:43.754374 23158 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776 [term 1 LEADER]: Becoming Leader. State: Replica: 18265af27667495a91630b5d11727776, State: Running, Role: LEADER
I20260620 20:12:43.754703 23158 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18265af27667495a91630b5d11727776" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40539 } }
I20260620 20:12:43.755160 23153 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:12:43.756184 23160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18265af27667495a91630b5d11727776. Latest consensus state: current_term: 1 leader_uuid: "18265af27667495a91630b5d11727776" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18265af27667495a91630b5d11727776" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40539 } } }
I20260620 20:12:43.756307 23160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776 [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:43.756556 23159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18265af27667495a91630b5d11727776" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18265af27667495a91630b5d11727776" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 40539 } } }
I20260620 20:12:43.756639 23159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776 [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:43.756680 23167 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:12:43.759506 23167 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:12:43.764614 23167 catalog_manager.cc:1347] Generated new cluster ID: 0f327d56ff434191a371f3581923da06
I20260620 20:12:43.764853 23167 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:12:43.777525 23167 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:12:43.778694 23167 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:12:43.784415 23167 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776: Generated new TSK 0
I20260620 20:12:43.785018 23167 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:12:43.793612 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:40539
--builtin_ntp_servers=127.5.91.148:33987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 20:12:43.901675 23177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:43.901906 23177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:43.901942 23177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:43.905824 23177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:43.905875 23177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:43.905929 23177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:12:43.910548 23177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:33987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:40539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:43.911571 23177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:43.912878 23177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:43.922413 23183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:43.922426 23185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:43.922431 23182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:43.922525 23177 server_base.cc:1034] running on GCE node
I20260620 20:12:43.923046 23177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:43.923703 23177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:43.924890 23177 hybrid_clock.cc:648] HybridClock initialized: now 1781986363924877 us; error 25 us; skew 500 ppm
I20260620 20:12:43.926817 23177 webserver.cc:460] Webserver started at http://127.5.91.129:42977/ using document root <none> and password file <none>
I20260620 20:12:43.927353 23177 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:43.927414 23177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:43.927623 23177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:43.929809 23177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "85a350338cf14ef38ee704a28233f6a9"
format_stamp: "Formatted at 2026-06-20 20:12:43 on dist-test-slave-l8h4"
server_key: "35cef3ba0eab08c3b5e14535a968ae23"
server_key_iv: "7a6d0784f90957bc7bbf1897b9b937e2"
server_key_version: "encryptionkey@0"
I20260620 20:12:43.930341 23177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "85a350338cf14ef38ee704a28233f6a9"
format_stamp: "Formatted at 2026-06-20 20:12:43 on dist-test-slave-l8h4"
server_key: "35cef3ba0eab08c3b5e14535a968ae23"
server_key_iv: "7a6d0784f90957bc7bbf1897b9b937e2"
server_key_version: "encryptionkey@0"
I20260620 20:12:43.933799 23177 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260620 20:12:43.936134 23191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:43.937347 23177 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 20:12:43.937445 23177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "85a350338cf14ef38ee704a28233f6a9"
format_stamp: "Formatted at 2026-06-20 20:12:43 on dist-test-slave-l8h4"
server_key: "35cef3ba0eab08c3b5e14535a968ae23"
server_key_iv: "7a6d0784f90957bc7bbf1897b9b937e2"
server_key_version: "encryptionkey@0"
I20260620 20:12:43.937551 23177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:43.951711 23177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:43.952334 23177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:43.952482 23177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:43.953029 23177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:43.953908 23177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:12:43.953960 23177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:43.954003 23177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:12:43.954032 23177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:43.962580 23177 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:34425
I20260620 20:12:43.962662 23305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:34425 every 8 connection(s)
I20260620 20:12:43.963456 23177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:12:43.970028 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 23177
I20260620 20:12:43.970191 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:12:43.970475 5486 external_mini_cluster.cc:1428] Setting key 1fe4d990248122e99fcb6f1f83428409
I20260620 20:12:43.972471 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:40539
--builtin_ntp_servers=127.5.91.148:33987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 20:12:43.972572 23156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:43.965249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:44025 (local address 127.5.91.190:40539)
0620 20:12:43.965426 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:43.965430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:43.966430 (+ 1000us) server_negotiation.cc:407] Connection header received
0620 20:12:43.967781 (+ 1351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:43.967784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:43.967830 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:43.967901 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:43.968761 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.969304 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.970524 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.970830 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.970944 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:43.972115 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:43.972119 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:43.972122 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:12:43.972135 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:43.972157 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:12:43.972167 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:43.972239 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:43.972354 (+ 115us) server_negotiation.cc:299] Negotiation successful
0620 20:12:43.972419 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260620 20:12:43.973359 23308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:43.965607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:40539 (local address 127.5.91.129:44025)
0620 20:12:43.966291 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:43.966329 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:43.967578 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:43.968048 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:43.968054 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:43.968132 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:43.968615 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:43.968628 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.969424 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.969428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:43.970401 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:43.970411 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:43.970989 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:43.970995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:43.971179 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:43.971837 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 20:12:43.971839 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:43.971890 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 20:12:43.971968 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:43.972239 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:43.972245 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:43.972248 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 20:12:43.972331 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 20:12:43.972546 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":109,"threads_started":1}
I20260620 20:12:43.974483 23306 heartbeater.cc:347] Connected to a master server at 127.5.91.190:40539
I20260620 20:12:43.974721 23306 heartbeater.cc:464] Registering TS with master...
I20260620 20:12:43.975303 23306 heartbeater.cc:511] Master 127.5.91.190:40539 requested a full tablet report, sending...
I20260620 20:12:43.976750 23117 ts_manager.cc:194] Registered new tserver with Master: 85a350338cf14ef38ee704a28233f6a9 (127.5.91.129:34425)
I20260620 20:12:43.977819 23117 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.91.129:44025
W20260620 20:12:44.079447 23309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:44.079684 23309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:44.079720 23309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:44.083756 23309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:44.083817 23309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:44.083870 23309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:12:44.088617 23309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:33987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:40539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:44.089685 23309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:44.091025 23309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:44.100867 23315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:44.100929 23314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:44.100875 23317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:44.101527 23309 server_base.cc:1034] running on GCE node
I20260620 20:12:44.101907 23309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:44.102586 23309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:44.103756 23309 hybrid_clock.cc:648] HybridClock initialized: now 1781986364103732 us; error 36 us; skew 500 ppm
I20260620 20:12:44.105645 23309 webserver.cc:460] Webserver started at http://127.5.91.130:42959/ using document root <none> and password file <none>
I20260620 20:12:44.106249 23309 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:44.106312 23309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:44.106494 23309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:44.108482 23309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "9ccfc41f7801438d8239d5507319c90d"
format_stamp: "Formatted at 2026-06-20 20:12:44 on dist-test-slave-l8h4"
server_key: "f8c3e212638b62ad475f60e2d08bb243"
server_key_iv: "b4efecfbb79140b9a08810a5f49f89cc"
server_key_version: "encryptionkey@0"
I20260620 20:12:44.109005 23309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "9ccfc41f7801438d8239d5507319c90d"
format_stamp: "Formatted at 2026-06-20 20:12:44 on dist-test-slave-l8h4"
server_key: "f8c3e212638b62ad475f60e2d08bb243"
server_key_iv: "b4efecfbb79140b9a08810a5f49f89cc"
server_key_version: "encryptionkey@0"
I20260620 20:12:44.112571 23309 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 20:12:44.115073 23323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:44.116312 23309 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:12:44.116415 23309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "9ccfc41f7801438d8239d5507319c90d"
format_stamp: "Formatted at 2026-06-20 20:12:44 on dist-test-slave-l8h4"
server_key: "f8c3e212638b62ad475f60e2d08bb243"
server_key_iv: "b4efecfbb79140b9a08810a5f49f89cc"
server_key_version: "encryptionkey@0"
I20260620 20:12:44.116537 23309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:44.129943 23309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:44.130611 23309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:44.130801 23309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:44.131368 23309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:44.132288 23309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:12:44.132334 23309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:44.132375 23309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:12:44.132403 23309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:44.140455 23309 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:40775
I20260620 20:12:44.140499 23437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:40775 every 8 connection(s)
I20260620 20:12:44.141358 23309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:12:44.149308 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 23309
I20260620 20:12:44.149438 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:12:44.149715 5486 external_mini_cluster.cc:1428] Setting key d2e9c83849a148876d754ac8faa19869
I20260620 20:12:44.150578 23156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:44.143154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:42539 (local address 127.5.91.190:40539)
0620 20:12:44.143304 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:44.143307 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:44.144263 (+ 956us) server_negotiation.cc:407] Connection header received
0620 20:12:44.145656 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:44.145660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:44.145722 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:44.145791 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:44.146803 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.147328 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.148598 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.148817 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.148904 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:44.150099 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:44.150103 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:44.150106 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:12:44.150120 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:44.150162 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:12:44.150174 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:44.150233 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:44.150370 (+ 137us) server_negotiation.cc:299] Negotiation successful
0620 20:12:44.150439 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260620 20:12:44.151422 23440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:44.143490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:40539 (local address 127.5.91.130:42539)
0620 20:12:44.144117 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:44.144155 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:44.145448 (+ 1293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:44.146058 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:44.146066 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:44.146177 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:44.146664 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:44.146677 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.147446 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.147450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:44.148462 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:44.148472 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.148956 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.148960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:44.149124 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:44.149835 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 20:12:44.149838 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:44.149879 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 20:12:44.149948 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:44.150245 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:44.150254 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:44.150259 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 20:12:44.150362 (+ 103us) client_negotiation.cc:240] Negotiation successful
0620 20:12:44.150641 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":105,"threads_started":1}
I20260620 20:12:44.151523 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:40539
--builtin_ntp_servers=127.5.91.148:33987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 20:12:44.152441 23438 heartbeater.cc:347] Connected to a master server at 127.5.91.190:40539
I20260620 20:12:44.152652 23438 heartbeater.cc:464] Registering TS with master...
I20260620 20:12:44.153126 23438 heartbeater.cc:511] Master 127.5.91.190:40539 requested a full tablet report, sending...
I20260620 20:12:44.154060 23117 ts_manager.cc:194] Registered new tserver with Master: 9ccfc41f7801438d8239d5507319c90d (127.5.91.130:40775)
I20260620 20:12:44.154838 23117 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.91.130:42539
W20260620 20:12:44.258471 23441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:44.258718 23441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:44.258754 23441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:44.263000 23441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:44.263058 23441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:44.263113 23441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:12:44.267791 23441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:33987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:40539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:44.268738 23441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:44.269989 23441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:44.278963 23446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:44.278980 23449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:44.278983 23447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:44.279229 23441 server_base.cc:1034] running on GCE node
I20260620 20:12:44.279623 23441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:44.280187 23441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:44.281354 23441 hybrid_clock.cc:648] HybridClock initialized: now 1781986364281323 us; error 41 us; skew 500 ppm
I20260620 20:12:44.283409 23441 webserver.cc:460] Webserver started at http://127.5.91.131:45795/ using document root <none> and password file <none>
I20260620 20:12:44.284155 23441 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:44.284222 23441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:44.284413 23441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:44.286448 23441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "ad17a18473ca4ed39277777e43736dee"
format_stamp: "Formatted at 2026-06-20 20:12:44 on dist-test-slave-l8h4"
server_key: "d708a11ad3e0e66e87224e68fb9d3d2a"
server_key_iv: "30986ec011028dc0147c51587a5b08d5"
server_key_version: "encryptionkey@0"
I20260620 20:12:44.286955 23441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "ad17a18473ca4ed39277777e43736dee"
format_stamp: "Formatted at 2026-06-20 20:12:44 on dist-test-slave-l8h4"
server_key: "d708a11ad3e0e66e87224e68fb9d3d2a"
server_key_iv: "30986ec011028dc0147c51587a5b08d5"
server_key_version: "encryptionkey@0"
I20260620 20:12:44.290525 23441 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:12:44.293002 23455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:44.294231 23441 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:12:44.294337 23441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "ad17a18473ca4ed39277777e43736dee"
format_stamp: "Formatted at 2026-06-20 20:12:44 on dist-test-slave-l8h4"
server_key: "d708a11ad3e0e66e87224e68fb9d3d2a"
server_key_iv: "30986ec011028dc0147c51587a5b08d5"
server_key_version: "encryptionkey@0"
I20260620 20:12:44.294445 23441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:44.311512 23441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:44.312182 23441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:44.312348 23441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:44.312911 23441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:44.313827 23441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:12:44.313872 23441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:44.313910 23441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:12:44.313925 23441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:44.322522 23441 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:40539
I20260620 20:12:44.322553 23569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:40539 every 8 connection(s)
I20260620 20:12:44.323393 23441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:12:44.327849 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 23441
I20260620 20:12:44.327970 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:12:44.328265 5486 external_mini_cluster.cc:1428] Setting key fd228b30f9cacc44ad086442d1b71700
I20260620 20:12:44.335439 23156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:44.325956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:39943 (local address 127.5.91.190:40539)
0620 20:12:44.326113 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:44.326116 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:44.327283 (+ 1167us) server_negotiation.cc:407] Connection header received
0620 20:12:44.328967 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:44.328971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:44.329028 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:44.329126 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:44.330747 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.331318 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.332915 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.333160 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.333258 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:44.334914 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:44.334917 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:44.334919 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:12:44.334932 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:44.334954 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:12:44.334964 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:44.335033 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:44.335249 (+ 216us) server_negotiation.cc:299] Negotiation successful
0620 20:12:44.335318 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 20:12:44.336328 23572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:44.326406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:40539 (local address 127.5.91.131:39943)
0620 20:12:44.327117 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:44.327163 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:44.328700 (+ 1537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:44.329568 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:44.329579 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:44.329724 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:44.330536 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:44.330559 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.331499 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.331502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:44.332723 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:44.332740 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.333361 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.333365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:44.333568 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:44.334605 (+ 1037us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 20:12:44.334607 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:44.334658 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 20:12:44.334743 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:44.335076 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:44.335082 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:44.335086 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 20:12:44.335180 (+ 94us) client_negotiation.cc:240] Negotiation successful
0620 20:12:44.335464 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":121,"threads_started":1}
I20260620 20:12:44.337558 23570 heartbeater.cc:347] Connected to a master server at 127.5.91.190:40539
I20260620 20:12:44.337807 23570 heartbeater.cc:464] Registering TS with master...
I20260620 20:12:44.338377 23570 heartbeater.cc:511] Master 127.5.91.190:40539 requested a full tablet report, sending...
I20260620 20:12:44.339402 23117 ts_manager.cc:194] Registered new tserver with Master: ad17a18473ca4ed39277777e43736dee (127.5.91.131:40539)
I20260620 20:12:44.339967 23117 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.91.131:39943
I20260620 20:12:44.343120 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 20:12:44.348898 23156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:44.345097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48814 (local address 127.5.91.190:40539)
0620 20:12:44.345213 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:44.345216 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:44.345406 (+ 190us) server_negotiation.cc:407] Connection header received
0620 20:12:44.345485 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:44.345487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:44.345542 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:44.345605 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:44.345911 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.346489 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.347358 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.347595 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.347675 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:44.348459 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:44.348462 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:44.348464 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:12:44.348476 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:44.348495 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:12:44.348504 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:44.348569 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:44.348716 (+ 147us) server_negotiation.cc:299] Negotiation successful
0620 20:12:44.348772 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 20:12:44.351656 23117 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48814:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:12:44.355711 23117 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:12:44.369800 23582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:44.363966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35720 (local address 127.5.91.129:34425)
0620 20:12:44.364824 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:44.364838 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:44.364867 (+ 29us) server_negotiation.cc:407] Connection header received
0620 20:12:44.364950 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:44.364956 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:44.365163 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:44.365341 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:44.365602 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.366669 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.368083 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.368874 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.368983 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:44.369154 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:44.369507 (+ 353us) server_negotiation.cc:299] Negotiation successful
0620 20:12:44.369627 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752,"thread_start_us":146,"threads_started":1}
I20260620 20:12:44.369793 23581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:44.363850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:34425 (local address 127.0.0.1:35720)
0620 20:12:44.364176 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:44.364219 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:44.364335 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:44.365324 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:44.365331 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:44.365353 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:44.365496 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:44.365504 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.366786 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.366789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:44.367938 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:44.367948 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.369063 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.369069 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:44.369299 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:44.369354 (+ 55us) client_negotiation.cc:769] Sending connection context
0620 20:12:44.369482 (+ 128us) client_negotiation.cc:240] Negotiation successful
0620 20:12:44.369640 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":71,"threads_started":1}
I20260620 20:12:44.371052 23585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:44.364087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:40539 (local address 127.0.0.1:44704)
0620 20:12:44.365016 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:44.365030 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:44.365116 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:44.365965 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:44.365968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:44.365986 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:44.366112 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:44.366120 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.367728 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.367732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:44.368865 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:44.368874 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.370590 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.370594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:44.370735 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:44.370760 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 20:12:44.370827 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 20:12:44.370908 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":867,"mutex_wait_us":66,"thread_start_us":105,"threads_started":1}
I20260620 20:12:44.371104 23584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:44.364632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44704 (local address 127.5.91.131:40539)
0620 20:12:44.365542 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:44.365554 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:44.365577 (+ 23us) server_negotiation.cc:407] Connection header received
0620 20:12:44.365656 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:44.365661 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:44.365832 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:44.365975 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:44.366277 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.367282 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.369583 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.370463 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.370568 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:44.370741 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:44.370864 (+ 123us) server_negotiation.cc:299] Negotiation successful
0620 20:12:44.370950 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":808,"thread_start_us":92,"threads_started":1}
I20260620 20:12:44.371148 23586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:44.364927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55274 (local address 127.5.91.130:40775)
0620 20:12:44.365468 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:44.365481 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:44.365700 (+ 219us) server_negotiation.cc:407] Connection header received
0620 20:12:44.365812 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:44.365817 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:44.365988 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:44.366120 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:44.366474 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.367298 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.369542 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.370327 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.370428 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:44.370576 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:44.370898 (+ 322us) server_negotiation.cc:299] Negotiation successful
0620 20:12:44.370991 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":132,"threads_started":1}
I20260620 20:12:44.371537 23583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:44.364773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:40775 (local address 127.0.0.1:55274)
0620 20:12:44.365591 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:44.365605 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:44.365707 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:44.366224 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:44.366227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:44.366246 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:44.366354 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:44.366361 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.367406 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.367409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:44.369405 (+ 1996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:44.369414 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.370624 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.370627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:44.370752 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:44.370771 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 20:12:44.370827 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 20:12:44.371450 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"mutex_wait_us":608,"thread_start_us":120,"threads_started":1}
I20260620 20:12:44.373121 23240 tablet_service.cc:1467] Processing CreateTablet for tablet aad1b23d27004896a1d45d2a0155d8df (DEFAULT_TABLE table=test-table [id=7305e014a7bf43f6adb81b3ddea3f66a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:12:44.374286 23240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aad1b23d27004896a1d45d2a0155d8df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:44.374168 23503 tablet_service.cc:1467] Processing CreateTablet for tablet aad1b23d27004896a1d45d2a0155d8df (DEFAULT_TABLE table=test-table [id=7305e014a7bf43f6adb81b3ddea3f66a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:12:44.375094 23503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aad1b23d27004896a1d45d2a0155d8df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:44.375681 23371 tablet_service.cc:1467] Processing CreateTablet for tablet aad1b23d27004896a1d45d2a0155d8df (DEFAULT_TABLE table=test-table [id=7305e014a7bf43f6adb81b3ddea3f66a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:12:44.376405 23371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aad1b23d27004896a1d45d2a0155d8df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:44.381661 23587 tablet_bootstrap.cc:492] T aad1b23d27004896a1d45d2a0155d8df P 85a350338cf14ef38ee704a28233f6a9: Bootstrap starting.
I20260620 20:12:44.382067 23588 tablet_bootstrap.cc:492] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee: Bootstrap starting.
I20260620 20:12:44.382726 23589 tablet_bootstrap.cc:492] T aad1b23d27004896a1d45d2a0155d8df P 9ccfc41f7801438d8239d5507319c90d: Bootstrap starting.
I20260620 20:12:44.384752 23589 tablet_bootstrap.cc:654] T aad1b23d27004896a1d45d2a0155d8df P 9ccfc41f7801438d8239d5507319c90d: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:44.384752 23587 tablet_bootstrap.cc:654] T aad1b23d27004896a1d45d2a0155d8df P 85a350338cf14ef38ee704a28233f6a9: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:44.385608 23589 log.cc:824] T aad1b23d27004896a1d45d2a0155d8df P 9ccfc41f7801438d8239d5507319c90d: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:44.385608 23587 log.cc:824] T aad1b23d27004896a1d45d2a0155d8df P 85a350338cf14ef38ee704a28233f6a9: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:44.387686 23587 tablet_bootstrap.cc:492] T aad1b23d27004896a1d45d2a0155d8df P 85a350338cf14ef38ee704a28233f6a9: No bootstrap required, opened a new log
I20260620 20:12:44.387688 23589 tablet_bootstrap.cc:492] T aad1b23d27004896a1d45d2a0155d8df P 9ccfc41f7801438d8239d5507319c90d: No bootstrap required, opened a new log
I20260620 20:12:44.387840 23588 tablet_bootstrap.cc:654] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:44.387876 23589 ts_tablet_manager.cc:1397] T aad1b23d27004896a1d45d2a0155d8df P 9ccfc41f7801438d8239d5507319c90d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 20:12:44.387955 23587 ts_tablet_manager.cc:1397] T aad1b23d27004896a1d45d2a0155d8df P 85a350338cf14ef38ee704a28233f6a9: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260620 20:12:44.388650 23588 log.cc:824] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:44.390560 23588 tablet_bootstrap.cc:492] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee: No bootstrap required, opened a new log
I20260620 20:12:44.390735 23588 ts_tablet_manager.cc:1397] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260620 20:12:44.391294 23589 raft_consensus.cc:348] T aad1b23d27004896a1d45d2a0155d8df P 9ccfc41f7801438d8239d5507319c90d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85a350338cf14ef38ee704a28233f6a9" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34425 } } peers { permanent_uuid: "ad17a18473ca4ed39277777e43736dee" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 40539 } } peers { permanent_uuid: "9ccfc41f7801438d8239d5507319c90d" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40775 } }
I20260620 20:12:44.391511 23589 raft_consensus.cc:374] T aad1b23d27004896a1d45d2a0155d8df P 9ccfc41f7801438d8239d5507319c90d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:44.391578 23589 raft_consensus.cc:729] T aad1b23d27004896a1d45d2a0155d8df P 9ccfc41f7801438d8239d5507319c90d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ccfc41f7801438d8239d5507319c90d, State: Initialized, Role: FOLLOWER
I20260620 20:12:44.391855 23587 raft_consensus.cc:348] T aad1b23d27004896a1d45d2a0155d8df P 85a350338cf14ef38ee704a28233f6a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85a350338cf14ef38ee704a28233f6a9" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34425 } } peers { permanent_uuid: "ad17a18473ca4ed39277777e43736dee" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 40539 } } peers { permanent_uuid: "9ccfc41f7801438d8239d5507319c90d" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40775 } }
I20260620 20:12:44.391996 23589 consensus_queue.cc:260] T aad1b23d27004896a1d45d2a0155d8df P 9ccfc41f7801438d8239d5507319c90d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85a350338cf14ef38ee704a28233f6a9" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34425 } } peers { permanent_uuid: "ad17a18473ca4ed39277777e43736dee" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 40539 } } peers { permanent_uuid: "9ccfc41f7801438d8239d5507319c90d" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40775 } }
I20260620 20:12:44.392206 23587 raft_consensus.cc:374] T aad1b23d27004896a1d45d2a0155d8df P 85a350338cf14ef38ee704a28233f6a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:44.392271 23587 raft_consensus.cc:729] T aad1b23d27004896a1d45d2a0155d8df P 85a350338cf14ef38ee704a28233f6a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85a350338cf14ef38ee704a28233f6a9, State: Initialized, Role: FOLLOWER
I20260620 20:12:44.392700 23587 consensus_queue.cc:260] T aad1b23d27004896a1d45d2a0155d8df P 85a350338cf14ef38ee704a28233f6a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85a350338cf14ef38ee704a28233f6a9" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34425 } } peers { permanent_uuid: "ad17a18473ca4ed39277777e43736dee" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 40539 } } peers { permanent_uuid: "9ccfc41f7801438d8239d5507319c90d" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40775 } }
I20260620 20:12:44.392886 23589 ts_tablet_manager.cc:1428] T aad1b23d27004896a1d45d2a0155d8df P 9ccfc41f7801438d8239d5507319c90d: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260620 20:12:44.393273 23438 heartbeater.cc:503] Master 127.5.91.190:40539 was elected leader, sending a full tablet report...
I20260620 20:12:44.393618 23587 ts_tablet_manager.cc:1428] T aad1b23d27004896a1d45d2a0155d8df P 85a350338cf14ef38ee704a28233f6a9: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260620 20:12:44.393927 23306 heartbeater.cc:503] Master 127.5.91.190:40539 was elected leader, sending a full tablet report...
I20260620 20:12:44.394688 23588 raft_consensus.cc:348] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85a350338cf14ef38ee704a28233f6a9" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34425 } } peers { permanent_uuid: "ad17a18473ca4ed39277777e43736dee" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 40539 } } peers { permanent_uuid: "9ccfc41f7801438d8239d5507319c90d" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40775 } }
I20260620 20:12:44.394915 23588 raft_consensus.cc:374] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:44.395001 23588 raft_consensus.cc:729] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad17a18473ca4ed39277777e43736dee, State: Initialized, Role: FOLLOWER
I20260620 20:12:44.395375 23588 consensus_queue.cc:260] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85a350338cf14ef38ee704a28233f6a9" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34425 } } peers { permanent_uuid: "ad17a18473ca4ed39277777e43736dee" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 40539 } } peers { permanent_uuid: "9ccfc41f7801438d8239d5507319c90d" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40775 } }
I20260620 20:12:44.395959 23570 heartbeater.cc:503] Master 127.5.91.190:40539 was elected leader, sending a full tablet report...
I20260620 20:12:44.396238 23588 ts_tablet_manager.cc:1428] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260620 20:12:44.418535 23595 raft_consensus.cc:482] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:12:44.418772 23595 raft_consensus.cc:504] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85a350338cf14ef38ee704a28233f6a9" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34425 } } peers { permanent_uuid: "ad17a18473ca4ed39277777e43736dee" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 40539 } } peers { permanent_uuid: "9ccfc41f7801438d8239d5507319c90d" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40775 } }
I20260620 20:12:44.420264 23595 leader_election.cc:288] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85a350338cf14ef38ee704a28233f6a9 (127.5.91.129:34425), 9ccfc41f7801438d8239d5507319c90d (127.5.91.130:40775)
I20260620 20:12:44.425272 23572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:44.420445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:34425 (local address 127.5.91.131:39135)
0620 20:12:44.420651 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:44.420665 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:44.420752 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:44.421040 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:44.421044 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:44.421063 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:44.421190 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:44.421197 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.422499 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.422503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:44.423619 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:44.423628 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.424810 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.424813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:44.424927 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:44.424953 (+ 26us) client_negotiation.cc:769] Sending connection context
0620 20:12:44.425052 (+ 99us) client_negotiation.cc:240] Negotiation successful
0620 20:12:44.425136 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":24}
I20260620 20:12:44.425364 23582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:44.420656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:39135 (local address 127.5.91.129:34425)
0620 20:12:44.420814 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:44.420819 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:44.420833 (+ 14us) server_negotiation.cc:407] Connection header received
0620 20:12:44.420887 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:44.420890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:44.420941 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:44.421013 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:44.421447 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.422319 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.423880 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.424594 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.424704 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:44.424751 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:44.425121 (+ 370us) server_negotiation.cc:299] Negotiation successful
0620 20:12:44.425236 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260620 20:12:44.425627 23586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:44.420723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:51699 (local address 127.5.91.130:40775)
0620 20:12:44.420841 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:44.420845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:44.421005 (+ 160us) server_negotiation.cc:407] Connection header received
0620 20:12:44.421114 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:44.421117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:44.421163 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:44.421242 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:44.421986 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.422764 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.424333 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.425012 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.425121 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:44.425162 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:44.425431 (+ 269us) server_negotiation.cc:299] Negotiation successful
0620 20:12:44.425512 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 20:12:44.425997 23596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:44.420485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:40775 (local address 127.5.91.131:51699)
0620 20:12:44.420910 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:44.420922 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:44.421016 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:44.421446 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:44.421450 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:44.421493 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:44.421678 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:44.421686 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.423033 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.423036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:44.424092 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:44.424108 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.425161 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.425165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:44.425263 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:44.425286 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 20:12:44.425772 (+ 486us) client_negotiation.cc:240] Negotiation successful
0620 20:12:44.425858 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":1,"thread_start_us":162,"threads_started":1}
I20260620 20:12:44.426115 23260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aad1b23d27004896a1d45d2a0155d8df" candidate_uuid: "ad17a18473ca4ed39277777e43736dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85a350338cf14ef38ee704a28233f6a9" is_pre_election: true
I20260620 20:12:44.426522 23260 raft_consensus.cc:2454] T aad1b23d27004896a1d45d2a0155d8df P 85a350338cf14ef38ee704a28233f6a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad17a18473ca4ed39277777e43736dee in term 0.
I20260620 20:12:44.426610 23391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aad1b23d27004896a1d45d2a0155d8df" candidate_uuid: "ad17a18473ca4ed39277777e43736dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ccfc41f7801438d8239d5507319c90d" is_pre_election: true
I20260620 20:12:44.426883 23391 raft_consensus.cc:2454] T aad1b23d27004896a1d45d2a0155d8df P 9ccfc41f7801438d8239d5507319c90d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad17a18473ca4ed39277777e43736dee in term 0.
I20260620 20:12:44.427095 23459 leader_election.cc:302] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85a350338cf14ef38ee704a28233f6a9, ad17a18473ca4ed39277777e43736dee; no voters:
I20260620 20:12:44.427364 23595 raft_consensus.cc:2781] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:12:44.427444 23595 raft_consensus.cc:482] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:12:44.427496 23595 raft_consensus.cc:3037] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:44.428985 23595 raft_consensus.cc:504] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85a350338cf14ef38ee704a28233f6a9" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34425 } } peers { permanent_uuid: "ad17a18473ca4ed39277777e43736dee" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 40539 } } peers { permanent_uuid: "9ccfc41f7801438d8239d5507319c90d" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40775 } }
I20260620 20:12:44.429358 23595 leader_election.cc:288] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee [CANDIDATE]: Term 1 election: Requested vote from peers 85a350338cf14ef38ee704a28233f6a9 (127.5.91.129:34425), 9ccfc41f7801438d8239d5507319c90d (127.5.91.130:40775)
I20260620 20:12:44.430110 23391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aad1b23d27004896a1d45d2a0155d8df" candidate_uuid: "ad17a18473ca4ed39277777e43736dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ccfc41f7801438d8239d5507319c90d"
I20260620 20:12:44.430110 23260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aad1b23d27004896a1d45d2a0155d8df" candidate_uuid: "ad17a18473ca4ed39277777e43736dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85a350338cf14ef38ee704a28233f6a9"
I20260620 20:12:44.430392 23260 raft_consensus.cc:3037] T aad1b23d27004896a1d45d2a0155d8df P 85a350338cf14ef38ee704a28233f6a9 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:44.430390 23391 raft_consensus.cc:3037] T aad1b23d27004896a1d45d2a0155d8df P 9ccfc41f7801438d8239d5507319c90d [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:44.432042 23391 raft_consensus.cc:2454] T aad1b23d27004896a1d45d2a0155d8df P 9ccfc41f7801438d8239d5507319c90d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad17a18473ca4ed39277777e43736dee in term 1.
I20260620 20:12:44.432076 23260 raft_consensus.cc:2454] T aad1b23d27004896a1d45d2a0155d8df P 85a350338cf14ef38ee704a28233f6a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad17a18473ca4ed39277777e43736dee in term 1.
I20260620 20:12:44.432433 23456 leader_election.cc:302] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ccfc41f7801438d8239d5507319c90d, ad17a18473ca4ed39277777e43736dee; no voters:
I20260620 20:12:44.432634 23595 raft_consensus.cc:2781] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:12:44.432878 23595 raft_consensus.cc:686] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee [term 1 LEADER]: Becoming Leader. State: Replica: ad17a18473ca4ed39277777e43736dee, State: Running, Role: LEADER
I20260620 20:12:44.433264 23595 consensus_queue.cc:237] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85a350338cf14ef38ee704a28233f6a9" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34425 } } peers { permanent_uuid: "ad17a18473ca4ed39277777e43736dee" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 40539 } } peers { permanent_uuid: "9ccfc41f7801438d8239d5507319c90d" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40775 } }
I20260620 20:12:44.436865 23117 catalog_manager.cc:5495] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee reported cstate change: term changed from 0 to 1, leader changed from <none> to ad17a18473ca4ed39277777e43736dee (127.5.91.131). New cstate: current_term: 1 leader_uuid: "ad17a18473ca4ed39277777e43736dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85a350338cf14ef38ee704a28233f6a9" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad17a18473ca4ed39277777e43736dee" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 40539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9ccfc41f7801438d8239d5507319c90d" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40775 } health_report { overall_health: UNKNOWN } } }
I20260620 20:12:44.459849 23584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:44.455202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44708 (local address 127.5.91.131:40539)
0620 20:12:44.455325 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:44.455328 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:44.455342 (+ 14us) server_negotiation.cc:407] Connection header received
0620 20:12:44.455435 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:44.455438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:44.455491 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:44.455607 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:12:44.456047 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.456580 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.457772 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:44.457993 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:44.458108 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:44.458585 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:12:44.459262 (+ 677us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:12:44.459409 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:44.459627 (+ 218us) server_negotiation.cc:299] Negotiation successful
0620 20:12:44.459683 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260620 20:12:44.464653 23307 tablet.cc:2369] T aad1b23d27004896a1d45d2a0155d8df P 85a350338cf14ef38ee704a28233f6a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:12:44.465868 23391 raft_consensus.cc:1261] T aad1b23d27004896a1d45d2a0155d8df P 9ccfc41f7801438d8239d5507319c90d [term 1 FOLLOWER]: Refusing update from remote peer ad17a18473ca4ed39277777e43736dee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:12:44.465866 23260 raft_consensus.cc:1261] T aad1b23d27004896a1d45d2a0155d8df P 85a350338cf14ef38ee704a28233f6a9 [term 1 FOLLOWER]: Refusing update from remote peer ad17a18473ca4ed39277777e43736dee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:12:44.466554 23597 consensus_queue.cc:1035] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ccfc41f7801438d8239d5507319c90d" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:12:44.466729 23595 consensus_queue.cc:1035] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a350338cf14ef38ee704a28233f6a9" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:12:44.473714 23600 mvcc.cc:205] Tried to move back new op lower bound from 7299016148845371392 to 7299016148721315840. Current Snapshot: MvccSnapshot[applied={T|T < 7299016148845371392}]
I20260620 20:12:44.473843 23602 mvcc.cc:205] Tried to move back new op lower bound from 7299016148845371392 to 7299016148721315840. Current Snapshot: MvccSnapshot[applied={T|T < 7299016148845371392}]
I20260620 20:12:44.474694 23601 mvcc.cc:205] Tried to move back new op lower bound from 7299016148845371392 to 7299016148721315840. Current Snapshot: MvccSnapshot[applied={T|T < 7299016148845371392}]
I20260620 20:12:44.484140 23115 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:12:44.484333 23115 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:12:44.486771 23115 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18265af27667495a91630b5d11727776: Sending DeleteTablet for 3 replicas of tablet aad1b23d27004896a1d45d2a0155d8df
I20260620 20:12:44.487511 23240 tablet_service.cc:1514] Processing DeleteTablet for tablet aad1b23d27004896a1d45d2a0155d8df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:12:44 UTC) from {username='slave'} at 127.0.0.1:35720
I20260620 20:12:44.487510 23503 tablet_service.cc:1514] Processing DeleteTablet for tablet aad1b23d27004896a1d45d2a0155d8df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:12:44 UTC) from {username='slave'} at 127.0.0.1:44704
I20260620 20:12:44.487702 23371 tablet_service.cc:1514] Processing DeleteTablet for tablet aad1b23d27004896a1d45d2a0155d8df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:12:44 UTC) from {username='slave'} at 127.0.0.1:55274
I20260620 20:12:44.488020 23610 tablet_replica.cc:330] T aad1b23d27004896a1d45d2a0155d8df P 85a350338cf14ef38ee704a28233f6a9: stopping tablet replica
I20260620 20:12:44.488363 23612 tablet_replica.cc:330] T aad1b23d27004896a1d45d2a0155d8df P 9ccfc41f7801438d8239d5507319c90d: stopping tablet replica
I20260620 20:12:44.488358 23610 raft_consensus.cc:2229] T aad1b23d27004896a1d45d2a0155d8df P 85a350338cf14ef38ee704a28233f6a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:12:44.488544 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 23177
I20260620 20:12:44.488669 23610 raft_consensus.cc:2258] T aad1b23d27004896a1d45d2a0155d8df P 85a350338cf14ef38ee704a28233f6a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:12:44.488714 23612 raft_consensus.cc:2229] T aad1b23d27004896a1d45d2a0155d8df P 9ccfc41f7801438d8239d5507319c90d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:12:44.488977 23612 raft_consensus.cc:2258] T aad1b23d27004896a1d45d2a0155d8df P 9ccfc41f7801438d8239d5507319c90d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:12:44.488986 23611 tablet_replica.cc:330] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee: stopping tablet replica
I20260620 20:12:44.489223 23611 raft_consensus.cc:2229] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee [term 1 LEADER]: Raft consensus shutting down.
I20260620 20:12:44.489562 23611 raft_consensus.cc:2258] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:12:44.490561 23612 ts_tablet_manager.cc:1905] T aad1b23d27004896a1d45d2a0155d8df P 9ccfc41f7801438d8239d5507319c90d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:12:44.490907 23611 ts_tablet_manager.cc:1905] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:12:44.493249 23612 ts_tablet_manager.cc:1918] T aad1b23d27004896a1d45d2a0155d8df P 9ccfc41f7801438d8239d5507319c90d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:12:44.493415 23612 log.cc:1196] T aad1b23d27004896a1d45d2a0155d8df P 9ccfc41f7801438d8239d5507319c90d: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-1/wal/wals/aad1b23d27004896a1d45d2a0155d8df
I20260620 20:12:44.493499 23611 ts_tablet_manager.cc:1918] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:12:44.493680 23611 log.cc:1196] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/aad1b23d27004896a1d45d2a0155d8df
I20260620 20:12:44.493786 23612 ts_tablet_manager.cc:1939] T aad1b23d27004896a1d45d2a0155d8df P 9ccfc41f7801438d8239d5507319c90d: Deleting consensus metadata
I20260620 20:12:44.494030 23611 ts_tablet_manager.cc:1939] T aad1b23d27004896a1d45d2a0155d8df P ad17a18473ca4ed39277777e43736dee: Deleting consensus metadata
I20260620 20:12:44.495057 23101 catalog_manager.cc:4841] TS 9ccfc41f7801438d8239d5507319c90d (127.5.91.130:40775): tablet aad1b23d27004896a1d45d2a0155d8df (table test-table [id=7305e014a7bf43f6adb81b3ddea3f66a]) successfully deleted
I20260620 20:12:44.495060 23103 catalog_manager.cc:4841] TS ad17a18473ca4ed39277777e43736dee (127.5.91.131:40539): tablet aad1b23d27004896a1d45d2a0155d8df (table test-table [id=7305e014a7bf43f6adb81b3ddea3f66a]) successfully deleted
W20260620 20:12:44.497294 23104 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.91.129:34425 (error 108)
I20260620 20:12:44.498015 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 23309
I20260620 20:12:44.498268 23583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:44.497972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:34425 (local address 127.0.0.1:35732)
0620 20:12:44.498081 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:44.498183 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.129:34425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260620 20:12:44.498641 23104 catalog_manager.cc:4584] TS 85a350338cf14ef38ee704a28233f6a9 (127.5.91.129:34425): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet aad1b23d27004896a1d45d2a0155d8df: Network error: Client connection negotiation failed: client connection to 127.5.91.129:34425: connect: Connection refused (error 111)
I20260620 20:12:44.503575 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 23441
I20260620 20:12:44.508348 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 23086
2026-06-20T20:12:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (984 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:12:44 dist-test-slave-l8h4 krb5kdc[23616](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:12:44 dist-test-slave-l8h4 krb5kdc[23616](info): set up 2 sockets
Jun 20 20:12:44 dist-test-slave-l8h4 krb5kdc[23616](info): commencing operation
krb5kdc: starting...
W20260620 20:12:45.478705 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.951s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:12:45 dist-test-slave-l8h4 krb5kdc[23616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:12:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:12:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:12:45.593305 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:41031
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:33529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:41031
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:12:45.701380 23629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:45.701645 23629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:45.701699 23629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:45.705838 23629 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:12:45.705950 23629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:45.706022 23629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:45.706059 23629 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:12:45.706084 23629 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:12:45.711354 23629 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:33529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:41031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:41031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:45.712304 23629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:45.713220 23629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:45.721213 23634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:45.721287 23635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:45.721328 23629 server_base.cc:1034] running on GCE node
W20260620 20:12:45.721550 23637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:45.722086 23629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:45.723186 23629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:45.724351 23629 hybrid_clock.cc:648] HybridClock initialized: now 1781986365724331 us; error 31 us; skew 500 ppm
Jun 20 20:12:45 dist-test-slave-l8h4 krb5kdc[23616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986365, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:45.727602 23629 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:12:45.728458 23629 webserver.cc:460] Webserver started at http://127.5.91.190:39523/ using document root <none> and password file <none>
I20260620 20:12:45.728994 23629 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:45.729059 23629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:45.729254 23629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:45.731248 23629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "93cb6a1b4ee146719c724e31e7914595"
format_stamp: "Formatted at 2026-06-20 20:12:45 on dist-test-slave-l8h4"
server_key: "bf53cfa02f1361ecff8f89fcdba1957f"
server_key_iv: "9b6cae6582b4ba5713240714ed74d355"
server_key_version: "encryptionkey@0"
I20260620 20:12:45.731762 23629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "93cb6a1b4ee146719c724e31e7914595"
format_stamp: "Formatted at 2026-06-20 20:12:45 on dist-test-slave-l8h4"
server_key: "bf53cfa02f1361ecff8f89fcdba1957f"
server_key_iv: "9b6cae6582b4ba5713240714ed74d355"
server_key_version: "encryptionkey@0"
I20260620 20:12:45.735512 23629 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260620 20:12:45.738459 23644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:45.739899 23629 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:12:45.740024 23629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "93cb6a1b4ee146719c724e31e7914595"
format_stamp: "Formatted at 2026-06-20 20:12:45 on dist-test-slave-l8h4"
server_key: "bf53cfa02f1361ecff8f89fcdba1957f"
server_key_iv: "9b6cae6582b4ba5713240714ed74d355"
server_key_version: "encryptionkey@0"
I20260620 20:12:45.740134 23629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:45.749465 23629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:45.752313 23629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:45.752471 23629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:45.759434 23629 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:41031
I20260620 20:12:45.759505 23696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:41031 every 8 connection(s)
I20260620 20:12:45.760386 23629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:12:45.763078 23697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:45.768208 23697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595: Bootstrap starting.
I20260620 20:12:45.769345 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 23629
I20260620 20:12:45.769474 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:12:45.769753 5486 external_mini_cluster.cc:1428] Setting key 9579e58a05394bc6d5a5a3d6f18bbf55
I20260620 20:12:45.770870 23697 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:45.771730 23697 log.cc:824] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:45.773717 23697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595: No bootstrap required, opened a new log
I20260620 20:12:45.778075 23697 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93cb6a1b4ee146719c724e31e7914595" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41031 } }
I20260620 20:12:45.778352 23697 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:45.778412 23697 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93cb6a1b4ee146719c724e31e7914595, State: Initialized, Role: FOLLOWER
I20260620 20:12:45.778937 23697 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93cb6a1b4ee146719c724e31e7914595" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41031 } }
I20260620 20:12:45.779101 23697 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:12:45.779163 23697 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:12:45.779258 23697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595 [term 0 FOLLOWER]: Advancing to term 1
Jun 20 20:12:45 dist-test-slave-l8h4 krb5kdc[23616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:12:45.780241 23697 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93cb6a1b4ee146719c724e31e7914595" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41031 } }
I20260620 20:12:45.780591 23697 leader_election.cc:302] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93cb6a1b4ee146719c724e31e7914595; no voters:
I20260620 20:12:45.780916 23697 leader_election.cc:288] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:12:45.781032 23702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:12:45.781241 23702 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595 [term 1 LEADER]: Becoming Leader. State: Replica: 93cb6a1b4ee146719c724e31e7914595, State: Running, Role: LEADER
I20260620 20:12:45.781570 23702 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93cb6a1b4ee146719c724e31e7914595" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41031 } }
I20260620 20:12:45.782083 23697 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:12:45.782786 23702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93cb6a1b4ee146719c724e31e7914595" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93cb6a1b4ee146719c724e31e7914595" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41031 } } }
I20260620 20:12:45.782920 23702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595 [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:45.783125 23702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93cb6a1b4ee146719c724e31e7914595. Latest consensus state: current_term: 1 leader_uuid: "93cb6a1b4ee146719c724e31e7914595" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93cb6a1b4ee146719c724e31e7914595" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41031 } } }
I20260620 20:12:45.783217 23702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595 [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:45.783496 23711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:12:45.786782 23711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:12:45.793246 23700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:45.770842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45852 (local address 127.5.91.190:41031)
0620 20:12:45.771254 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:45.771271 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:45.771311 (+ 40us) server_negotiation.cc:407] Connection header received
0620 20:12:45.772134 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:45.772151 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:45.772462 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:45.772802 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:45.773431 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:45.774910 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:45.776195 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:45.777235 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:45.777990 (+ 755us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:45.785024 (+ 7034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:45.785039 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:45.785074 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:45.785106 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:45.790179 (+ 5073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:45.790740 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:45.790746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:45.790752 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:45.790902 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:45.791229 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:45.791232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:45.791234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:45.791754 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:45.791943 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:45.792227 (+ 284us) server_negotiation.cc:299] Negotiation successful
0620 20:12:45.792534 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":130,"threads_started":1}
I20260620 20:12:45.796562 23711 catalog_manager.cc:1347] Generated new cluster ID: a4d7474ae8724f0f86f90784d31ef38a
I20260620 20:12:45.796625 23711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:12:45.805722 23711 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:12:45.806911 23711 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:12:45.819706 23711 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595: Generated new TSK 0
I20260620 20:12:45.820480 23711 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.91.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
I20260620 20:12:45.850035 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41031
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:33529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:12:45.962306 23723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:45.962610 23723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:45.962668 23723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:45.966778 23723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:45.966841 23723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:45.966922 23723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:12:45.972254 23723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:33529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:45.973312 23723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:45.974316 23723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:45.983832 23731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:45.983866 23728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:45.983865 23729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:45.984472 23723 server_base.cc:1034] running on GCE node
I20260620 20:12:45.984892 23723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:45.985566 23723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:45.986843 23723 hybrid_clock.cc:648] HybridClock initialized: now 1781986365986815 us; error 38 us; skew 500 ppm
Jun 20 20:12:45 dist-test-slave-l8h4 krb5kdc[23616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986365, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:45.990116 23723 init.cc:377] Logged in from keytab as kudu/127.5.91.129@KRBTEST.COM (short username kudu)
I20260620 20:12:45.991076 23723 webserver.cc:460] Webserver started at http://127.5.91.129:36883/ using document root <none> and password file <none>
I20260620 20:12:45.991609 23723 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:45.991672 23723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:45.991842 23723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:45.993860 23723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "3c95d8c286ac4b049021e23c5ed433b2"
format_stamp: "Formatted at 2026-06-20 20:12:45 on dist-test-slave-l8h4"
server_key: "42a064cee37a4df932723062d3ca72de"
server_key_iv: "c896d41963b8ebe7ecefa7933742ced4"
server_key_version: "encryptionkey@0"
I20260620 20:12:45.994432 23723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "3c95d8c286ac4b049021e23c5ed433b2"
format_stamp: "Formatted at 2026-06-20 20:12:45 on dist-test-slave-l8h4"
server_key: "42a064cee37a4df932723062d3ca72de"
server_key_iv: "c896d41963b8ebe7ecefa7933742ced4"
server_key_version: "encryptionkey@0"
I20260620 20:12:45.998037 23723 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:12:46.001691 23738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:46.003430 23723 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:12:46.003559 23723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "3c95d8c286ac4b049021e23c5ed433b2"
format_stamp: "Formatted at 2026-06-20 20:12:45 on dist-test-slave-l8h4"
server_key: "42a064cee37a4df932723062d3ca72de"
server_key_iv: "c896d41963b8ebe7ecefa7933742ced4"
server_key_version: "encryptionkey@0"
I20260620 20:12:46.003676 23723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:46.027637 23723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:46.030711 23723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:46.030862 23723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:46.031419 23723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:46.032192 23723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:12:46.032238 23723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:46.032275 23723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:12:46.032292 23723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:46.040478 23723 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:34235
I20260620 20:12:46.040544 23851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:34235 every 8 connection(s)
I20260620 20:12:46.041491 23723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:12:46.047241 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 23723
I20260620 20:12:46.047360 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:12:46.047621 5486 external_mini_cluster.cc:1428] Setting key 688a4ee4c95067d318581a48f9e058f4
Jun 20 20:12:46 dist-test-slave-l8h4 krb5kdc[23616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986365, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:12:46.056221 23700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:46.043402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:50263 (local address 127.5.91.190:41031)
0620 20:12:46.043569 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:46.043574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:46.044378 (+ 804us) server_negotiation.cc:407] Connection header received
0620 20:12:46.045636 (+ 1258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:46.045640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:46.045692 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:46.045776 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:46.047023 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.047687 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.049025 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.049265 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.049351 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:46.052287 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:46.052290 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:46.052292 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:46.052318 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:46.053885 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:46.054512 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:46.054515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:46.054516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:46.054594 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:46.055001 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:46.055005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:46.055007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:46.055240 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:46.055324 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:46.055957 (+ 633us) server_negotiation.cc:299] Negotiation successful
0620 20:12:46.056084 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
WARNING: no policy specified for kudu/127.5.91.130@KRBTEST.COM; defaulting to no policy
I20260620 20:12:46.057245 23854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:46.043733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41031 (local address 127.5.91.129:50263)
0620 20:12:46.044243 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:46.044285 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:46.045421 (+ 1136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:46.045954 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:46.045961 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:46.046370 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:46.046850 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:46.046864 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.047827 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.047830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:46.048890 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:46.048900 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.049418 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.049422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:46.049606 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:46.050245 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:12:46.050264 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:46.052099 (+ 1835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:46.054008 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:46.054014 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:46.054030 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:46.054415 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:46.054679 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:46.054683 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:46.054685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:46.054862 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:46.055347 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:46.055354 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:46.055682 (+ 328us) client_negotiation.cc:769] Sending connection context
0620 20:12:46.055887 (+ 205us) client_negotiation.cc:240] Negotiation successful
0620 20:12:46.056221 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":111,"threads_started":1}
I20260620 20:12:46.058750 23852 heartbeater.cc:347] Connected to a master server at 127.5.91.190:41031
I20260620 20:12:46.059015 23852 heartbeater.cc:464] Registering TS with master...
I20260620 20:12:46.059526 23852 heartbeater.cc:511] Master 127.5.91.190:41031 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.130@KRBTEST.COM" created.
I20260620 20:12:46.060994 23660 ts_manager.cc:194] Registered new tserver with Master: 3c95d8c286ac4b049021e23c5ed433b2 (127.5.91.129:34235)
I20260620 20:12:46.062089 23660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:50263
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
I20260620 20:12:46.075459 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41031
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:33529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:12:46.183071 23857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:46.183377 23857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:46.183437 23857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:46.187412 23857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:46.187463 23857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:46.187520 23857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:12:46.193117 23857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:33529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:46.194120 23857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:46.195127 23857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:46.204195 23863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:46.204201 23865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:46.204201 23862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:46.204589 23857 server_base.cc:1034] running on GCE node
I20260620 20:12:46.204968 23857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:46.205547 23857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:46.206717 23857 hybrid_clock.cc:648] HybridClock initialized: now 1781986366206702 us; error 26 us; skew 500 ppm
Jun 20 20:12:46 dist-test-slave-l8h4 krb5kdc[23616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986366, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:46.209369 23857 init.cc:377] Logged in from keytab as kudu/127.5.91.130@KRBTEST.COM (short username kudu)
I20260620 20:12:46.210237 23857 webserver.cc:460] Webserver started at http://127.5.91.130:33723/ using document root <none> and password file <none>
I20260620 20:12:46.210721 23857 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:46.210776 23857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:46.210939 23857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:46.212850 23857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "9dd1a28f21564b978f0c3677cccf9736"
format_stamp: "Formatted at 2026-06-20 20:12:46 on dist-test-slave-l8h4"
server_key: "7ca0112c28a131e0f0b516ed3887726d"
server_key_iv: "6c216a784a77b0ed353d7d9190ae2639"
server_key_version: "encryptionkey@0"
I20260620 20:12:46.213332 23857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "9dd1a28f21564b978f0c3677cccf9736"
format_stamp: "Formatted at 2026-06-20 20:12:46 on dist-test-slave-l8h4"
server_key: "7ca0112c28a131e0f0b516ed3887726d"
server_key_iv: "6c216a784a77b0ed353d7d9190ae2639"
server_key_version: "encryptionkey@0"
I20260620 20:12:46.216764 23857 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 20:12:46.219210 23872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:46.220526 23857 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:12:46.220649 23857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "9dd1a28f21564b978f0c3677cccf9736"
format_stamp: "Formatted at 2026-06-20 20:12:46 on dist-test-slave-l8h4"
server_key: "7ca0112c28a131e0f0b516ed3887726d"
server_key_iv: "6c216a784a77b0ed353d7d9190ae2639"
server_key_version: "encryptionkey@0"
I20260620 20:12:46.220768 23857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:46.270980 23857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:46.274158 23857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:46.274334 23857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:46.274922 23857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:46.275774 23857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:12:46.275820 23857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:46.275858 23857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:12:46.275877 23857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:46.284516 23857 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:40473
I20260620 20:12:46.284796 23985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:40473 every 8 connection(s)
I20260620 20:12:46.285418 23857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:12:46.292737 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 23857
I20260620 20:12:46.292867 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:12:46.293129 5486 external_mini_cluster.cc:1428] Setting key 568a3b06028b1bcada9f3cc712ad5847
Jun 20 20:12:46 dist-test-slave-l8h4 krb5kdc[23616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986366, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.91.131@KRBTEST.COM; defaulting to no policy
I20260620 20:12:46.303520 23700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:46.287327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:35215 (local address 127.5.91.190:41031)
0620 20:12:46.287462 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:46.287465 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:46.288282 (+ 817us) server_negotiation.cc:407] Connection header received
0620 20:12:46.289531 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:46.289534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:46.289581 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:46.289652 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:46.290858 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.291414 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.292525 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.292754 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.292844 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:46.299302 (+ 6458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:46.299306 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:46.299309 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:46.299341 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:46.301199 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:46.301769 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:46.301773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:46.301775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:46.301865 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:46.302310 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:46.302314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:46.302316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:46.302570 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:46.302671 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:46.303268 (+ 597us) server_negotiation.cc:299] Negotiation successful
0620 20:12:46.303379 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 20:12:46.304338 23988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:46.287706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41031 (local address 127.5.91.130:35215)
0620 20:12:46.288132 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:46.288173 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:46.289327 (+ 1154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:46.289823 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:46.289829 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:46.290207 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:46.290700 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:46.290714 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.291539 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.291542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:46.292406 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:46.292415 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.292896 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.292900 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:46.293071 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:46.297336 (+ 4265us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:12:46.297359 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:46.299073 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:46.301355 (+ 2282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:46.301361 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:46.301375 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:46.301659 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:46.301967 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:46.301970 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:46.301971 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:46.302215 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:46.302693 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:46.302699 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:46.302981 (+ 282us) client_negotiation.cc:769] Sending connection context
0620 20:12:46.303175 (+ 194us) client_negotiation.cc:240] Negotiation successful
0620 20:12:46.303453 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":104,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.131@KRBTEST.COM" created.
I20260620 20:12:46.305469 23986 heartbeater.cc:347] Connected to a master server at 127.5.91.190:41031
I20260620 20:12:46.305743 23986 heartbeater.cc:464] Registering TS with master...
I20260620 20:12:46.306277 23986 heartbeater.cc:511] Master 127.5.91.190:41031 requested a full tablet report, sending...
I20260620 20:12:46.307387 23660 ts_manager.cc:194] Registered new tserver with Master: 9dd1a28f21564b978f0c3677cccf9736 (127.5.91.130:40473)
I20260620 20:12:46.308044 23660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:35215
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
I20260620 20:12:46.321769 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41031
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:33529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:12:46.433612 23991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:46.433884 23991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:46.433943 23991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:46.437911 23991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:46.437973 23991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:46.438059 23991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:12:46.443249 23991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:33529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:41031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:46.444305 23991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:46.445276 23991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:46.454478 23997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:46.454715 23991 server_base.cc:1034] running on GCE node
W20260620 20:12:46.454522 23996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:46.454522 23999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:46.455511 23991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:46.456238 23991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:46.457433 23991 hybrid_clock.cc:648] HybridClock initialized: now 1781986366457410 us; error 38 us; skew 500 ppm
Jun 20 20:12:46 dist-test-slave-l8h4 krb5kdc[23616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986366, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:46.460836 23991 init.cc:377] Logged in from keytab as kudu/127.5.91.131@KRBTEST.COM (short username kudu)
I20260620 20:12:46.461858 23991 webserver.cc:460] Webserver started at http://127.5.91.131:38993/ using document root <none> and password file <none>
I20260620 20:12:46.462452 23991 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:46.462528 23991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:46.462726 23991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:46.464731 23991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "0bcd5b1e2cda43edbf99499f7240f774"
format_stamp: "Formatted at 2026-06-20 20:12:46 on dist-test-slave-l8h4"
server_key: "9028f3d558768ff16f7cadc4da743436"
server_key_iv: "9fe65ac2dfabe0669f257c6b6dcc8229"
server_key_version: "encryptionkey@0"
I20260620 20:12:46.465258 23991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "0bcd5b1e2cda43edbf99499f7240f774"
format_stamp: "Formatted at 2026-06-20 20:12:46 on dist-test-slave-l8h4"
server_key: "9028f3d558768ff16f7cadc4da743436"
server_key_iv: "9fe65ac2dfabe0669f257c6b6dcc8229"
server_key_version: "encryptionkey@0"
I20260620 20:12:46.469331 23991 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260620 20:12:46.473006 24006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:46.474692 23991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:12:46.474829 23991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "0bcd5b1e2cda43edbf99499f7240f774"
format_stamp: "Formatted at 2026-06-20 20:12:46 on dist-test-slave-l8h4"
server_key: "9028f3d558768ff16f7cadc4da743436"
server_key_iv: "9fe65ac2dfabe0669f257c6b6dcc8229"
server_key_version: "encryptionkey@0"
I20260620 20:12:46.474951 23991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:46.488445 23991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:46.491748 23991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:46.491947 23991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:46.492560 23991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:46.493415 23991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:12:46.493467 23991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:46.493525 23991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:12:46.493556 23991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:46.503552 23991 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:32787
I20260620 20:12:46.503589 24119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:32787 every 8 connection(s)
I20260620 20:12:46.504434 23991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:12:46.508581 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 23991
I20260620 20:12:46.508723 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:12:46.508975 5486 external_mini_cluster.cc:1428] Setting key ba02d9ff725ca5db455687eef05e1e1c
Jun 20 20:12:46 dist-test-slave-l8h4 krb5kdc[23616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986366, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:12:46.518081 23700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:46.506234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:50355 (local address 127.5.91.190:41031)
0620 20:12:46.506414 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:46.506418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:46.507240 (+ 822us) server_negotiation.cc:407] Connection header received
0620 20:12:46.508525 (+ 1285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:46.508528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:46.508589 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:46.508704 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:46.510100 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.510887 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.512061 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.512286 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.512374 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:46.514747 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:46.514750 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:46.514752 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:46.514777 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:46.516128 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:46.516610 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:46.516613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:46.516614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:46.516694 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:46.517061 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:46.517064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:46.517065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:46.517274 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:46.517360 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:46.517852 (+ 492us) server_negotiation.cc:299] Negotiation successful
0620 20:12:46.517958 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260620 20:12:46.518944 24122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:46.506637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:41031 (local address 127.5.91.131:50355)
0620 20:12:46.507090 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:46.507133 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:46.508299 (+ 1166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:46.508937 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:46.508978 (+ 41us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:46.509381 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:46.509965 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:46.509981 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.511045 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.511048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:46.511937 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:46.511956 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.512417 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.512420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:46.512591 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:46.513136 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:12:46.513152 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:46.514572 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:46.516236 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:46.516240 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:46.516251 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:46.516500 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:46.516804 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:46.516806 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:46.516808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:46.516943 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:46.517378 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:46.517384 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:46.517617 (+ 233us) client_negotiation.cc:769] Sending connection context
0620 20:12:46.517780 (+ 163us) client_negotiation.cc:240] Negotiation successful
0620 20:12:46.518052 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":105,"threads_started":1}
I20260620 20:12:46.520143 24120 heartbeater.cc:347] Connected to a master server at 127.5.91.190:41031
I20260620 20:12:46.520349 24120 heartbeater.cc:464] Registering TS with master...
I20260620 20:12:46.520817 24120 heartbeater.cc:511] Master 127.5.91.190:41031 requested a full tablet report, sending...
I20260620 20:12:46.521809 23660 ts_manager.cc:194] Registered new tserver with Master: 0bcd5b1e2cda43edbf99499f7240f774 (127.5.91.131:32787)
I20260620 20:12:46.522441 23660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:50355
I20260620 20:12:46.522939 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 20:12:46.534291 23700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:46.525311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45858 (local address 127.5.91.190:41031)
0620 20:12:46.525457 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:46.525461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:46.525970 (+ 509us) server_negotiation.cc:407] Connection header received
0620 20:12:46.526065 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:46.526068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:46.526118 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:46.526205 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:46.526910 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.527415 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.528199 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.528408 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.528491 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:46.530648 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:46.530652 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:46.530655 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:46.530695 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:46.532317 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:46.532957 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:46.532959 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:46.532961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:46.533087 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:46.533405 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:46.533408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:46.533409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:46.533620 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:46.533762 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:46.534009 (+ 247us) server_negotiation.cc:299] Negotiation successful
0620 20:12:46.534121 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 20:12:46.537678 23660 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45858:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:12:46.540694 23660 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:12:46.554709 24132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:46.549135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:32787 (local address 127.0.0.1:42394)
0620 20:12:46.550281 (+ 1146us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:46.550293 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:46.550383 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:46.550921 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:46.550927 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:46.550947 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:46.551085 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:46.551092 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.552137 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.552140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:46.553263 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:46.553272 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.554022 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.554025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:46.554262 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:46.554308 (+ 46us) client_negotiation.cc:769] Sending connection context
0620 20:12:46.554416 (+ 108us) client_negotiation.cc:240] Negotiation successful
0620 20:12:46.554547 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1048,"mutex_wait_us":55,"thread_start_us":78,"threads_started":1}
I20260620 20:12:46.554730 24134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:46.549692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42394 (local address 127.5.91.131:32787)
0620 20:12:46.550059 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:46.550069 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:46.550471 (+ 402us) server_negotiation.cc:407] Connection header received
0620 20:12:46.550558 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:46.550564 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:46.550769 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:46.550916 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:46.551188 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.552029 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.553375 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.553915 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.554007 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:46.554189 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:46.554465 (+ 276us) server_negotiation.cc:299] Negotiation successful
0620 20:12:46.554571 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":171,"threads_started":1}
I20260620 20:12:46.555805 24131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:46.549082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:40473 (local address 127.0.0.1:56044)
0620 20:12:46.549832 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:46.549870 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:46.549999 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:46.551190 (+ 1191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:46.551193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:46.551210 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:46.551328 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:46.551335 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.552834 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.552837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:46.554217 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:46.554228 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.555377 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.555380 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:46.555493 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:46.555547 (+ 54us) client_negotiation.cc:769] Sending connection context
0620 20:12:46.555611 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 20:12:46.555679 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"mutex_wait_us":58,"thread_start_us":82,"threads_started":1}
I20260620 20:12:46.556092 24136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:46.550430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37166 (local address 127.5.91.129:34235)
0620 20:12:46.550848 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:46.550872 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:46.550891 (+ 19us) server_negotiation.cc:407] Connection header received
0620 20:12:46.550960 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:46.550966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:46.551196 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:46.551355 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:46.551534 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.552686 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.554639 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.555367 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.555465 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:46.555591 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:46.555716 (+ 125us) server_negotiation.cc:299] Negotiation successful
0620 20:12:46.555959 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":86,"threads_started":1}
I20260620 20:12:46.555929 24135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:46.550181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:34235 (local address 127.0.0.1:37166)
0620 20:12:46.550626 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:46.550638 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:46.550709 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:46.551331 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:46.551334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:46.551358 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:46.551434 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:46.551440 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.552830 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.552833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:46.554499 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:46.554508 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.555467 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.555471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:46.555591 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:46.555613 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 20:12:46.555682 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 20:12:46.555754 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":455,"thread_start_us":101,"threads_started":1}
I20260620 20:12:46.557127 24133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:46.549333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56044 (local address 127.5.91.130:40473)
0620 20:12:46.549635 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:46.549647 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:46.550486 (+ 839us) server_negotiation.cc:407] Connection header received
0620 20:12:46.550575 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:46.550580 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:46.550772 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:46.550918 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:46.551482 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.552679 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.554340 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.555113 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.555214 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:46.555319 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:46.556853 (+ 1534us) server_negotiation.cc:299] Negotiation successful
0620 20:12:46.556959 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":105,"threads_started":1}
I20260620 20:12:46.559221 24054 tablet_service.cc:1467] Processing CreateTablet for tablet a4d792ea4419477f858e952464624575 (DEFAULT_TABLE table=test-table [id=ee25372a5b324281b63e2e8afcf877c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:12:46.559484 23786 tablet_service.cc:1467] Processing CreateTablet for tablet a4d792ea4419477f858e952464624575 (DEFAULT_TABLE table=test-table [id=ee25372a5b324281b63e2e8afcf877c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:12:46.560195 24054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4d792ea4419477f858e952464624575. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:46.560441 23786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4d792ea4419477f858e952464624575. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:46.560125 23920 tablet_service.cc:1467] Processing CreateTablet for tablet a4d792ea4419477f858e952464624575 (DEFAULT_TABLE table=test-table [id=ee25372a5b324281b63e2e8afcf877c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:12:46.561010 23920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4d792ea4419477f858e952464624575. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:46.566500 24137 tablet_bootstrap.cc:492] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774: Bootstrap starting.
I20260620 20:12:46.567823 24138 tablet_bootstrap.cc:492] T a4d792ea4419477f858e952464624575 P 3c95d8c286ac4b049021e23c5ed433b2: Bootstrap starting.
I20260620 20:12:46.568370 24139 tablet_bootstrap.cc:492] T a4d792ea4419477f858e952464624575 P 9dd1a28f21564b978f0c3677cccf9736: Bootstrap starting.
I20260620 20:12:46.568750 24137 tablet_bootstrap.cc:654] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:46.569660 24138 tablet_bootstrap.cc:654] T a4d792ea4419477f858e952464624575 P 3c95d8c286ac4b049021e23c5ed433b2: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:46.570034 24137 log.cc:824] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:46.570341 24138 log.cc:824] T a4d792ea4419477f858e952464624575 P 3c95d8c286ac4b049021e23c5ed433b2: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:46.570412 24139 tablet_bootstrap.cc:654] T a4d792ea4419477f858e952464624575 P 9dd1a28f21564b978f0c3677cccf9736: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:46.571504 24139 log.cc:824] T a4d792ea4419477f858e952464624575 P 9dd1a28f21564b978f0c3677cccf9736: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:46.572124 24137 tablet_bootstrap.cc:492] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774: No bootstrap required, opened a new log
I20260620 20:12:46.572299 24137 ts_tablet_manager.cc:1397] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260620 20:12:46.572366 24138 tablet_bootstrap.cc:492] T a4d792ea4419477f858e952464624575 P 3c95d8c286ac4b049021e23c5ed433b2: No bootstrap required, opened a new log
I20260620 20:12:46.572535 24138 ts_tablet_manager.cc:1397] T a4d792ea4419477f858e952464624575 P 3c95d8c286ac4b049021e23c5ed433b2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 20:12:46.575845 24138 raft_consensus.cc:348] T a4d792ea4419477f858e952464624575 P 3c95d8c286ac4b049021e23c5ed433b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dd1a28f21564b978f0c3677cccf9736" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40473 } } peers { permanent_uuid: "0bcd5b1e2cda43edbf99499f7240f774" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 32787 } } peers { permanent_uuid: "3c95d8c286ac4b049021e23c5ed433b2" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34235 } }
I20260620 20:12:46.576076 24138 raft_consensus.cc:374] T a4d792ea4419477f858e952464624575 P 3c95d8c286ac4b049021e23c5ed433b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:46.576133 24138 raft_consensus.cc:729] T a4d792ea4419477f858e952464624575 P 3c95d8c286ac4b049021e23c5ed433b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c95d8c286ac4b049021e23c5ed433b2, State: Initialized, Role: FOLLOWER
I20260620 20:12:46.576469 24139 tablet_bootstrap.cc:492] T a4d792ea4419477f858e952464624575 P 9dd1a28f21564b978f0c3677cccf9736: No bootstrap required, opened a new log
I20260620 20:12:46.576553 24138 consensus_queue.cc:260] T a4d792ea4419477f858e952464624575 P 3c95d8c286ac4b049021e23c5ed433b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dd1a28f21564b978f0c3677cccf9736" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40473 } } peers { permanent_uuid: "0bcd5b1e2cda43edbf99499f7240f774" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 32787 } } peers { permanent_uuid: "3c95d8c286ac4b049021e23c5ed433b2" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34235 } }
I20260620 20:12:46.576334 24137 raft_consensus.cc:348] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dd1a28f21564b978f0c3677cccf9736" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40473 } } peers { permanent_uuid: "0bcd5b1e2cda43edbf99499f7240f774" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 32787 } } peers { permanent_uuid: "3c95d8c286ac4b049021e23c5ed433b2" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34235 } }
I20260620 20:12:46.576790 24137 raft_consensus.cc:374] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:46.576930 24137 raft_consensus.cc:729] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bcd5b1e2cda43edbf99499f7240f774, State: Initialized, Role: FOLLOWER
I20260620 20:12:46.576946 24139 ts_tablet_manager.cc:1397] T a4d792ea4419477f858e952464624575 P 9dd1a28f21564b978f0c3677cccf9736: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260620 20:12:46.577271 23852 heartbeater.cc:503] Master 127.5.91.190:41031 was elected leader, sending a full tablet report...
I20260620 20:12:46.577553 24138 ts_tablet_manager.cc:1428] T a4d792ea4419477f858e952464624575 P 3c95d8c286ac4b049021e23c5ed433b2: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260620 20:12:46.577684 24137 consensus_queue.cc:260] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dd1a28f21564b978f0c3677cccf9736" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40473 } } peers { permanent_uuid: "0bcd5b1e2cda43edbf99499f7240f774" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 32787 } } peers { permanent_uuid: "3c95d8c286ac4b049021e23c5ed433b2" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34235 } }
I20260620 20:12:46.579020 24137 ts_tablet_manager.cc:1428] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774: Time spent starting tablet: real 0.007s user 0.006s sys 0.002s
I20260620 20:12:46.579455 24120 heartbeater.cc:503] Master 127.5.91.190:41031 was elected leader, sending a full tablet report...
I20260620 20:12:46.582670 24139 raft_consensus.cc:348] T a4d792ea4419477f858e952464624575 P 9dd1a28f21564b978f0c3677cccf9736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dd1a28f21564b978f0c3677cccf9736" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40473 } } peers { permanent_uuid: "0bcd5b1e2cda43edbf99499f7240f774" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 32787 } } peers { permanent_uuid: "3c95d8c286ac4b049021e23c5ed433b2" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34235 } }
I20260620 20:12:46.582967 24139 raft_consensus.cc:374] T a4d792ea4419477f858e952464624575 P 9dd1a28f21564b978f0c3677cccf9736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:46.583027 24139 raft_consensus.cc:729] T a4d792ea4419477f858e952464624575 P 9dd1a28f21564b978f0c3677cccf9736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dd1a28f21564b978f0c3677cccf9736, State: Initialized, Role: FOLLOWER
I20260620 20:12:46.583565 24139 consensus_queue.cc:260] T a4d792ea4419477f858e952464624575 P 9dd1a28f21564b978f0c3677cccf9736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dd1a28f21564b978f0c3677cccf9736" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40473 } } peers { permanent_uuid: "0bcd5b1e2cda43edbf99499f7240f774" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 32787 } } peers { permanent_uuid: "3c95d8c286ac4b049021e23c5ed433b2" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34235 } }
I20260620 20:12:46.584293 23986 heartbeater.cc:503] Master 127.5.91.190:41031 was elected leader, sending a full tablet report...
I20260620 20:12:46.584614 24139 ts_tablet_manager.cc:1428] T a4d792ea4419477f858e952464624575 P 9dd1a28f21564b978f0c3677cccf9736: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
W20260620 20:12:46.755455 24121 tablet.cc:2369] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 20:12:46.786502 23987 tablet.cc:2369] T a4d792ea4419477f858e952464624575 P 9dd1a28f21564b978f0c3677cccf9736: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 20:12:46.792711 23853 tablet.cc:2369] T a4d792ea4419477f858e952464624575 P 3c95d8c286ac4b049021e23c5ed433b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:12:46.825804 24144 raft_consensus.cc:482] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:12:46.826038 24144 raft_consensus.cc:504] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dd1a28f21564b978f0c3677cccf9736" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40473 } } peers { permanent_uuid: "0bcd5b1e2cda43edbf99499f7240f774" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 32787 } } peers { permanent_uuid: "3c95d8c286ac4b049021e23c5ed433b2" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34235 } }
I20260620 20:12:46.827380 24144 leader_election.cc:288] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9dd1a28f21564b978f0c3677cccf9736 (127.5.91.130:40473), 3c95d8c286ac4b049021e23c5ed433b2 (127.5.91.129:34235)
I20260620 20:12:46.831023 24122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:46.827635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:34235 (local address 127.5.91.131:53727)
0620 20:12:46.827821 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:46.827841 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:46.828008 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:46.828318 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:46.828321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:46.828339 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:46.828448 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:46.828453 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.829193 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.829196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:46.829982 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:46.829991 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.830633 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.830637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:46.830737 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:46.830756 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 20:12:46.830819 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 20:12:46.830889 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 20:12:46.831023 24136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:46.827742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:53727 (local address 127.5.91.129:34235)
0620 20:12:46.827859 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:46.827864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:46.827938 (+ 74us) server_negotiation.cc:407] Connection header received
0620 20:12:46.828146 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:46.828149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:46.828197 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:46.828282 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:46.828557 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.829085 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.830103 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.830525 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.830633 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:46.830687 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:46.830843 (+ 156us) server_negotiation.cc:299] Negotiation successful
0620 20:12:46.830915 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 20:12:46.831400 24133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:46.827742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:41583 (local address 127.5.91.130:40473)
0620 20:12:46.827866 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:46.827870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:46.828508 (+ 638us) server_negotiation.cc:407] Connection header received
0620 20:12:46.828601 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:46.828604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:46.828644 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:46.828705 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:46.828983 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.829486 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.830514 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.830884 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.830985 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:46.831032 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:46.831218 (+ 186us) server_negotiation.cc:299] Negotiation successful
0620 20:12:46.831289 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 20:12:46.831633 23806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4d792ea4419477f858e952464624575" candidate_uuid: "0bcd5b1e2cda43edbf99499f7240f774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c95d8c286ac4b049021e23c5ed433b2" is_pre_election: true
I20260620 20:12:46.831733 24146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:46.827635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:40473 (local address 127.5.91.131:41583)
0620 20:12:46.828418 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:46.828430 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:46.828517 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:46.828726 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:46.828728 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:46.828743 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:46.828853 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:46.828858 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.829590 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.829593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:46.830398 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:46.830407 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.830985 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.830989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:46.831082 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:46.831104 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 20:12:46.831534 (+ 430us) client_negotiation.cc:240] Negotiation successful
0620 20:12:46.831606 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"mutex_wait_us":452,"thread_start_us":118,"threads_started":1}
I20260620 20:12:46.831909 23806 raft_consensus.cc:2454] T a4d792ea4419477f858e952464624575 P 3c95d8c286ac4b049021e23c5ed433b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bcd5b1e2cda43edbf99499f7240f774 in term 0.
I20260620 20:12:46.832226 23940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4d792ea4419477f858e952464624575" candidate_uuid: "0bcd5b1e2cda43edbf99499f7240f774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dd1a28f21564b978f0c3677cccf9736" is_pre_election: true
I20260620 20:12:46.832355 24010 leader_election.cc:302] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bcd5b1e2cda43edbf99499f7240f774, 3c95d8c286ac4b049021e23c5ed433b2; no voters:
I20260620 20:12:46.832458 23940 raft_consensus.cc:2454] T a4d792ea4419477f858e952464624575 P 9dd1a28f21564b978f0c3677cccf9736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bcd5b1e2cda43edbf99499f7240f774 in term 0.
I20260620 20:12:46.832603 24144 raft_consensus.cc:2781] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:12:46.832677 24144 raft_consensus.cc:482] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:12:46.832724 24144 raft_consensus.cc:3037] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:46.833756 24144 raft_consensus.cc:504] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dd1a28f21564b978f0c3677cccf9736" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40473 } } peers { permanent_uuid: "0bcd5b1e2cda43edbf99499f7240f774" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 32787 } } peers { permanent_uuid: "3c95d8c286ac4b049021e23c5ed433b2" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34235 } }
I20260620 20:12:46.834098 24144 leader_election.cc:288] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774 [CANDIDATE]: Term 1 election: Requested vote from peers 9dd1a28f21564b978f0c3677cccf9736 (127.5.91.130:40473), 3c95d8c286ac4b049021e23c5ed433b2 (127.5.91.129:34235)
I20260620 20:12:46.834455 23940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4d792ea4419477f858e952464624575" candidate_uuid: "0bcd5b1e2cda43edbf99499f7240f774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dd1a28f21564b978f0c3677cccf9736"
I20260620 20:12:46.834476 23806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4d792ea4419477f858e952464624575" candidate_uuid: "0bcd5b1e2cda43edbf99499f7240f774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c95d8c286ac4b049021e23c5ed433b2"
I20260620 20:12:46.834565 23940 raft_consensus.cc:3037] T a4d792ea4419477f858e952464624575 P 9dd1a28f21564b978f0c3677cccf9736 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:46.834574 23806 raft_consensus.cc:3037] T a4d792ea4419477f858e952464624575 P 3c95d8c286ac4b049021e23c5ed433b2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:46.835543 23806 raft_consensus.cc:2454] T a4d792ea4419477f858e952464624575 P 3c95d8c286ac4b049021e23c5ed433b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bcd5b1e2cda43edbf99499f7240f774 in term 1.
I20260620 20:12:46.835549 23940 raft_consensus.cc:2454] T a4d792ea4419477f858e952464624575 P 9dd1a28f21564b978f0c3677cccf9736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bcd5b1e2cda43edbf99499f7240f774 in term 1.
I20260620 20:12:46.835805 24010 leader_election.cc:302] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bcd5b1e2cda43edbf99499f7240f774, 3c95d8c286ac4b049021e23c5ed433b2; no voters:
I20260620 20:12:46.835996 24144 raft_consensus.cc:2781] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:12:46.836187 24144 raft_consensus.cc:686] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774 [term 1 LEADER]: Becoming Leader. State: Replica: 0bcd5b1e2cda43edbf99499f7240f774, State: Running, Role: LEADER
I20260620 20:12:46.836519 24144 consensus_queue.cc:237] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dd1a28f21564b978f0c3677cccf9736" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40473 } } peers { permanent_uuid: "0bcd5b1e2cda43edbf99499f7240f774" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 32787 } } peers { permanent_uuid: "3c95d8c286ac4b049021e23c5ed433b2" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34235 } }
I20260620 20:12:46.840034 23661 catalog_manager.cc:5495] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bcd5b1e2cda43edbf99499f7240f774 (127.5.91.131). New cstate: current_term: 1 leader_uuid: "0bcd5b1e2cda43edbf99499f7240f774" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dd1a28f21564b978f0c3677cccf9736" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bcd5b1e2cda43edbf99499f7240f774" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 32787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c95d8c286ac4b049021e23c5ed433b2" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260620 20:12:46.920686 24134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:46.917210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42408 (local address 127.5.91.131:32787)
0620 20:12:46.917338 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:46.917342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:46.917364 (+ 22us) server_negotiation.cc:407] Connection header received
0620 20:12:46.917464 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:46.917467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:46.917524 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:46.917612 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:12:46.917843 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.918370 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.919241 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:46.919449 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:46.919534 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:46.919842 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:12:46.920247 (+ 405us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:12:46.920371 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:46.920495 (+ 124us) server_negotiation.cc:299] Negotiation successful
0620 20:12:46.920535 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 20:12:46.926438 23806 raft_consensus.cc:1261] T a4d792ea4419477f858e952464624575 P 3c95d8c286ac4b049021e23c5ed433b2 [term 1 FOLLOWER]: Refusing update from remote peer 0bcd5b1e2cda43edbf99499f7240f774: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:12:46.926438 23940 raft_consensus.cc:1261] T a4d792ea4419477f858e952464624575 P 9dd1a28f21564b978f0c3677cccf9736 [term 1 FOLLOWER]: Refusing update from remote peer 0bcd5b1e2cda43edbf99499f7240f774: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:12:46.927070 24144 consensus_queue.cc:1035] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dd1a28f21564b978f0c3677cccf9736" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:12:46.927230 24147 consensus_queue.cc:1035] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c95d8c286ac4b049021e23c5ed433b2" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:12:46.933662 24152 mvcc.cc:205] Tried to move back new op lower bound from 7299016158923190272 to 7299016158564839424. Current Snapshot: MvccSnapshot[applied={T|T < 7299016158923190272}]
I20260620 20:12:46.941919 23661 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:12:46.942094 23661 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:12:46.944411 23661 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 93cb6a1b4ee146719c724e31e7914595: Sending DeleteTablet for 3 replicas of tablet a4d792ea4419477f858e952464624575
I20260620 20:12:46.945067 23920 tablet_service.cc:1514] Processing DeleteTablet for tablet a4d792ea4419477f858e952464624575 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:12:46 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:56044
I20260620 20:12:46.945254 23786 tablet_service.cc:1514] Processing DeleteTablet for tablet a4d792ea4419477f858e952464624575 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:12:46 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:37166
I20260620 20:12:46.945323 24054 tablet_service.cc:1514] Processing DeleteTablet for tablet a4d792ea4419477f858e952464624575 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:12:46 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:42394
I20260620 20:12:46.945683 24160 tablet_replica.cc:330] T a4d792ea4419477f858e952464624575 P 9dd1a28f21564b978f0c3677cccf9736: stopping tablet replica
I20260620 20:12:46.946009 24160 raft_consensus.cc:2229] T a4d792ea4419477f858e952464624575 P 9dd1a28f21564b978f0c3677cccf9736 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:12:46.946053 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 23723
I20260620 20:12:46.946348 24160 raft_consensus.cc:2258] T a4d792ea4419477f858e952464624575 P 9dd1a28f21564b978f0c3677cccf9736 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:12:46.947935 24160 ts_tablet_manager.cc:1905] T a4d792ea4419477f858e952464624575 P 9dd1a28f21564b978f0c3677cccf9736: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:12:46.950317 24162 tablet_replica.cc:330] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774: stopping tablet replica
I20260620 20:12:46.950572 24160 ts_tablet_manager.cc:1918] T a4d792ea4419477f858e952464624575 P 9dd1a28f21564b978f0c3677cccf9736: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:12:46.950663 24160 log.cc:1196] T a4d792ea4419477f858e952464624575 P 9dd1a28f21564b978f0c3677cccf9736: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-1/wal/wals/a4d792ea4419477f858e952464624575
I20260620 20:12:46.950744 24162 raft_consensus.cc:2229] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774 [term 1 LEADER]: Raft consensus shutting down.
I20260620 20:12:46.950924 24160 ts_tablet_manager.cc:1939] T a4d792ea4419477f858e952464624575 P 9dd1a28f21564b978f0c3677cccf9736: Deleting consensus metadata
I20260620 20:12:46.951190 24162 raft_consensus.cc:2258] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:12:46.952039 23647 catalog_manager.cc:4841] TS 9dd1a28f21564b978f0c3677cccf9736 (127.5.91.130:40473): tablet a4d792ea4419477f858e952464624575 (table test-table [id=ee25372a5b324281b63e2e8afcf877c3]) successfully deleted
I20260620 20:12:46.952595 24162 ts_tablet_manager.cc:1905] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 20:12:46.954226 23648 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.91.129:34235 (error 108)
I20260620 20:12:46.955087 24135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:46.954839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:34235 (local address 127.0.0.1:37176)
0620 20:12:46.954947 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:46.955007 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.129:34235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260620 20:12:46.955117 24162 ts_tablet_manager.cc:1918] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:12:46.955180 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 23857
I20260620 20:12:46.955192 24162 log.cc:1196] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/a4d792ea4419477f858e952464624575
W20260620 20:12:46.955369 23648 catalog_manager.cc:4584] TS 3c95d8c286ac4b049021e23c5ed433b2 (127.5.91.129:34235): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a4d792ea4419477f858e952464624575: Network error: Client connection negotiation failed: client connection to 127.5.91.129:34235: connect: Connection refused (error 111)
I20260620 20:12:46.955473 24162 ts_tablet_manager.cc:1939] T a4d792ea4419477f858e952464624575 P 0bcd5b1e2cda43edbf99499f7240f774: Deleting consensus metadata
I20260620 20:12:46.956499 23645 catalog_manager.cc:4841] TS 0bcd5b1e2cda43edbf99499f7240f774 (127.5.91.131:32787): tablet a4d792ea4419477f858e952464624575 (table test-table [id=ee25372a5b324281b63e2e8afcf877c3]) successfully deleted
I20260620 20:12:46.960395 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 23991
I20260620 20:12:46.965163 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 23629
2026-06-20T20:12:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2457 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-20T20:12:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:12:46Z Disabled control of system clock
I20260620 20:12:46.993256 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:33385
--webserver_interface=127.5.91.190
--webserver_port=0
--builtin_ntp_servers=127.5.91.148:35315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:33385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 20:12:47.106033 24167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:47.106338 24167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:47.106374 24167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:47.110374 24167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:12:47.110430 24167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:47.110445 24167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:47.110459 24167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:12:47.110474 24167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:12:47.115198 24167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:35315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:33385
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:33385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:47.116230 24167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:47.117539 24167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 20:12:47.125636 24167 server_base.cc:1034] running on GCE node
W20260620 20:12:47.125567 24175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:47.125555 24172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:47.125559 24173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:47.126353 24167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:47.127372 24167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:47.128540 24167 hybrid_clock.cc:648] HybridClock initialized: now 1781986367128528 us; error 24 us; skew 500 ppm
I20260620 20:12:47.130424 24167 webserver.cc:460] Webserver started at http://127.5.91.190:41279/ using document root <none> and password file <none>
I20260620 20:12:47.130937 24167 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:47.130999 24167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:47.131182 24167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:47.133077 24167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "2b3ba43b66794b3abab8dcaa9007d68f"
format_stamp: "Formatted at 2026-06-20 20:12:47 on dist-test-slave-l8h4"
server_key: "d0c39cd05497f79ce9cea13a49816d0b"
server_key_iv: "b6667950d327a2b85fcba531f3b475a3"
server_key_version: "encryptionkey@0"
I20260620 20:12:47.133586 24167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "2b3ba43b66794b3abab8dcaa9007d68f"
format_stamp: "Formatted at 2026-06-20 20:12:47 on dist-test-slave-l8h4"
server_key: "d0c39cd05497f79ce9cea13a49816d0b"
server_key_iv: "b6667950d327a2b85fcba531f3b475a3"
server_key_version: "encryptionkey@0"
I20260620 20:12:47.136943 24167 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 20:12:47.139283 24181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:47.140450 24167 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 20:12:47.140559 24167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "2b3ba43b66794b3abab8dcaa9007d68f"
format_stamp: "Formatted at 2026-06-20 20:12:47 on dist-test-slave-l8h4"
server_key: "d0c39cd05497f79ce9cea13a49816d0b"
server_key_iv: "b6667950d327a2b85fcba531f3b475a3"
server_key_version: "encryptionkey@0"
I20260620 20:12:47.140672 24167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:47.174319 24167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:47.175418 24167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:47.175705 24167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:47.182881 24167 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:33385
I20260620 20:12:47.182894 24233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:33385 every 8 connection(s)
I20260620 20:12:47.183780 24167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:12:47.186731 24234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:47.189138 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 24167
I20260620 20:12:47.189257 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:12:47.189513 5486 external_mini_cluster.cc:1428] Setting key fae9b6fa7ebdddb6c3e48b1063ab4721
I20260620 20:12:47.192612 24234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f: Bootstrap starting.
I20260620 20:12:47.195174 24234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:47.195863 24234 log.cc:824] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:47.197799 24234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f: No bootstrap required, opened a new log
I20260620 20:12:47.199321 24237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:47.190863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58246 (local address 127.5.91.190:33385)
0620 20:12:47.191535 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:47.191553 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:47.191595 (+ 42us) server_negotiation.cc:407] Connection header received
0620 20:12:47.192367 (+ 772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:47.192389 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:47.192854 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:47.193199 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:47.193616 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.195038 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.196019 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.196669 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.196844 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:47.197699 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:47.197704 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:47.197716 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:12:47.198113 (+ 397us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:47.198196 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:12:47.198219 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:47.198303 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:47.198439 (+ 136us) server_negotiation.cc:299] Negotiation successful
0620 20:12:47.198686 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":131,"threads_started":1}
I20260620 20:12:47.201224 24234 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b3ba43b66794b3abab8dcaa9007d68f" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33385 } }
I20260620 20:12:47.201462 24234 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:47.201519 24234 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b3ba43b66794b3abab8dcaa9007d68f, State: Initialized, Role: FOLLOWER
I20260620 20:12:47.201915 24234 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b3ba43b66794b3abab8dcaa9007d68f" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33385 } }
I20260620 20:12:47.202037 24234 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:12:47.202100 24234 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:12:47.202214 24234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:47.203032 24234 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b3ba43b66794b3abab8dcaa9007d68f" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33385 } }
I20260620 20:12:47.203333 24234 leader_election.cc:302] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b3ba43b66794b3abab8dcaa9007d68f; no voters:
I20260620 20:12:47.203596 24234 leader_election.cc:288] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:12:47.203715 24239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:12:47.203929 24239 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f [term 1 LEADER]: Becoming Leader. State: Replica: 2b3ba43b66794b3abab8dcaa9007d68f, State: Running, Role: LEADER
I20260620 20:12:47.204222 24239 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b3ba43b66794b3abab8dcaa9007d68f" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33385 } }
I20260620 20:12:47.204698 24234 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:12:47.205641 24240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b3ba43b66794b3abab8dcaa9007d68f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b3ba43b66794b3abab8dcaa9007d68f" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33385 } } }
I20260620 20:12:47.205760 24240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:47.206264 24248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:12:47.206386 24241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b3ba43b66794b3abab8dcaa9007d68f. Latest consensus state: current_term: 1 leader_uuid: "2b3ba43b66794b3abab8dcaa9007d68f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b3ba43b66794b3abab8dcaa9007d68f" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 33385 } } }
I20260620 20:12:47.206516 24241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:47.209381 24248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:12:47.214205 24248 catalog_manager.cc:1347] Generated new cluster ID: f29b5d18d8b2472699c6d63eeff9c770
I20260620 20:12:47.214268 24248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:12:47.233764 24248 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:12:47.234745 24248 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:12:47.240355 24248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f: Generated new TSK 0
I20260620 20:12:47.241055 24248 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:12:47.243822 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:33385
--builtin_ntp_servers=127.5.91.148:35315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 20:12:47.373970 24258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:47.374245 24258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:47.374282 24258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:47.378203 24258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:47.378261 24258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:47.378315 24258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:12:47.382972 24258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:35315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:33385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:47.383967 24258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:47.385223 24258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:47.394498 24266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:47.394455 24264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:47.394487 24258 server_base.cc:1034] running on GCE node
W20260620 20:12:47.394462 24263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:47.395076 24258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:47.395654 24258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:47.396835 24258 hybrid_clock.cc:648] HybridClock initialized: now 1781986367396817 us; error 29 us; skew 500 ppm
I20260620 20:12:47.398705 24258 webserver.cc:460] Webserver started at http://127.5.91.129:35871/ using document root <none> and password file <none>
I20260620 20:12:47.399241 24258 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:47.399304 24258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:47.399482 24258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:47.401412 24258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "ca61bf94299b4bbf84ca4da1fcb7c616"
format_stamp: "Formatted at 2026-06-20 20:12:47 on dist-test-slave-l8h4"
server_key: "46501bb0940915f95eb94bc465f69d73"
server_key_iv: "df14fe1985fe94397e5d4ff6c093513a"
server_key_version: "encryptionkey@0"
I20260620 20:12:47.401921 24258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "ca61bf94299b4bbf84ca4da1fcb7c616"
format_stamp: "Formatted at 2026-06-20 20:12:47 on dist-test-slave-l8h4"
server_key: "46501bb0940915f95eb94bc465f69d73"
server_key_iv: "df14fe1985fe94397e5d4ff6c093513a"
server_key_version: "encryptionkey@0"
I20260620 20:12:47.405471 24258 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 20:12:47.407963 24272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:47.409176 24258 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260620 20:12:47.409286 24258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "ca61bf94299b4bbf84ca4da1fcb7c616"
format_stamp: "Formatted at 2026-06-20 20:12:47 on dist-test-slave-l8h4"
server_key: "46501bb0940915f95eb94bc465f69d73"
server_key_iv: "df14fe1985fe94397e5d4ff6c093513a"
server_key_version: "encryptionkey@0"
I20260620 20:12:47.409384 24258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:47.424935 24258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:47.425621 24258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:47.425792 24258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:47.426379 24258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:47.427301 24258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:12:47.427350 24258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:47.427402 24258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:12:47.427430 24258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:47.435961 24258 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:38427
I20260620 20:12:47.435982 24385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:38427 every 8 connection(s)
I20260620 20:12:47.436826 24258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:12:47.441599 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 24258
I20260620 20:12:47.441749 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:12:47.442080 5486 external_mini_cluster.cc:1428] Setting key 6c7a319abe233fd3749361ee4fdcb759
I20260620 20:12:47.444324 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:33385
--builtin_ntp_servers=127.5.91.148:35315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 20:12:47.447072 24237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:47.438664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:38441 (local address 127.5.91.190:33385)
0620 20:12:47.438806 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:47.438810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:47.439991 (+ 1181us) server_negotiation.cc:407] Connection header received
0620 20:12:47.441378 (+ 1387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:47.441380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:47.441429 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:47.441531 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:47.442712 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.443392 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.444968 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.445219 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.445314 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:47.446567 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:47.446571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:47.446574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:12:47.446590 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:47.446613 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:12:47.446622 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:47.446691 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:47.446848 (+ 157us) server_negotiation.cc:299] Negotiation successful
0620 20:12:47.446915 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 20:12:47.447981 24388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:47.439064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33385 (local address 127.5.91.129:38441)
0620 20:12:47.439781 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:47.439827 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:47.441152 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:47.441719 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:47.441729 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:47.441833 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:47.442554 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:47.442576 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.443509 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.443513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:47.444828 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:47.444843 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.445384 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.445389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:47.445603 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:47.446290 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 20:12:47.446291 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:47.446333 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 20:12:47.446406 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:47.446701 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:47.446712 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:47.446718 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 20:12:47.446840 (+ 122us) client_negotiation.cc:240] Negotiation successful
0620 20:12:47.447139 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":105,"threads_started":1}
I20260620 20:12:47.449276 24386 heartbeater.cc:347] Connected to a master server at 127.5.91.190:33385
I20260620 20:12:47.449518 24386 heartbeater.cc:464] Registering TS with master...
I20260620 20:12:47.450014 24386 heartbeater.cc:511] Master 127.5.91.190:33385 requested a full tablet report, sending...
I20260620 20:12:47.451601 24198 ts_manager.cc:194] Registered new tserver with Master: ca61bf94299b4bbf84ca4da1fcb7c616 (127.5.91.129:38427)
I20260620 20:12:47.453141 24198 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.91.129:38441
W20260620 20:12:47.554157 24389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:47.554399 24389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:47.554435 24389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:47.558367 24389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:47.558410 24389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:47.558461 24389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:12:47.563014 24389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:35315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:33385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:47.563979 24389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:47.565270 24389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:47.574416 24397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:47.574611 24389 server_base.cc:1034] running on GCE node
W20260620 20:12:47.574414 24394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:47.574414 24395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:47.575093 24389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:47.575738 24389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:47.576905 24389 hybrid_clock.cc:648] HybridClock initialized: now 1781986367576886 us; error 34 us; skew 500 ppm
I20260620 20:12:47.578922 24389 webserver.cc:460] Webserver started at http://127.5.91.130:43373/ using document root <none> and password file <none>
I20260620 20:12:47.579470 24389 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:47.579531 24389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:47.579725 24389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:47.582288 24389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "9572de4f62cf4a7aacabf6bfe5d5812e"
format_stamp: "Formatted at 2026-06-20 20:12:47 on dist-test-slave-l8h4"
server_key: "4eee68c8306b83378c498ae7da2e3d21"
server_key_iv: "61c0fb2fbaf897c63da779bb63037183"
server_key_version: "encryptionkey@0"
I20260620 20:12:47.583037 24389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "9572de4f62cf4a7aacabf6bfe5d5812e"
format_stamp: "Formatted at 2026-06-20 20:12:47 on dist-test-slave-l8h4"
server_key: "4eee68c8306b83378c498ae7da2e3d21"
server_key_iv: "61c0fb2fbaf897c63da779bb63037183"
server_key_version: "encryptionkey@0"
I20260620 20:12:47.586915 24389 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260620 20:12:47.589218 24403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:47.590600 24389 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:12:47.590715 24389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "9572de4f62cf4a7aacabf6bfe5d5812e"
format_stamp: "Formatted at 2026-06-20 20:12:47 on dist-test-slave-l8h4"
server_key: "4eee68c8306b83378c498ae7da2e3d21"
server_key_iv: "61c0fb2fbaf897c63da779bb63037183"
server_key_version: "encryptionkey@0"
I20260620 20:12:47.590837 24389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:47.607679 24389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:47.608350 24389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:47.608501 24389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:47.609042 24389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:47.609921 24389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:12:47.609967 24389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:47.610006 24389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:12:47.610033 24389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:47.618507 24389 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:35585
I20260620 20:12:47.618592 24516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:35585 every 8 connection(s)
I20260620 20:12:47.619401 24389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:12:47.620846 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 24389
I20260620 20:12:47.620942 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:12:47.621291 5486 external_mini_cluster.cc:1428] Setting key 64c442e21a41a91da663a0cdf004170b
I20260620 20:12:47.623150 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:33385
--builtin_ntp_servers=127.5.91.148:35315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 20:12:47.628631 24237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:47.621177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:53963 (local address 127.5.91.190:33385)
0620 20:12:47.621324 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:47.621328 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:47.622459 (+ 1131us) server_negotiation.cc:407] Connection header received
0620 20:12:47.624075 (+ 1616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:47.624078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:47.624132 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:47.624209 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:47.625211 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.625730 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.626889 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.627103 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.627189 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:47.628184 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:47.628187 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:47.628189 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:12:47.628200 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:47.628217 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:12:47.628224 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:47.628281 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:47.628409 (+ 128us) server_negotiation.cc:299] Negotiation successful
0620 20:12:47.628469 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 20:12:47.629405 24519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:47.621582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33385 (local address 127.5.91.130:53963)
0620 20:12:47.622312 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:47.622360 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:47.623870 (+ 1510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:47.624392 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:47.624399 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:47.624489 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:47.625051 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:47.625064 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.625845 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.625848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:47.626777 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:47.626787 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.627237 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.627241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:47.627393 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:47.627938 (+ 545us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 20:12:47.627940 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:47.627977 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 20:12:47.628037 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:47.628290 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:47.628298 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:47.628303 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 20:12:47.628404 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 20:12:47.628669 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":122,"threads_started":1}
I20260620 20:12:47.630477 24517 heartbeater.cc:347] Connected to a master server at 127.5.91.190:33385
I20260620 20:12:47.630697 24517 heartbeater.cc:464] Registering TS with master...
I20260620 20:12:47.631166 24517 heartbeater.cc:511] Master 127.5.91.190:33385 requested a full tablet report, sending...
I20260620 20:12:47.632122 24198 ts_manager.cc:194] Registered new tserver with Master: 9572de4f62cf4a7aacabf6bfe5d5812e (127.5.91.130:35585)
I20260620 20:12:47.632891 24198 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.91.130:53963
W20260620 20:12:47.732336 24520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:47.732570 24520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:47.732606 24520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:47.736469 24520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:47.736512 24520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:47.736565 24520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:12:47.741226 24520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:35315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:33385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:47.742220 24520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:47.743512 24520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:47.753060 24526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:47.753103 24525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:47.753079 24528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:47.753294 24520 server_base.cc:1034] running on GCE node
I20260620 20:12:47.753746 24520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:47.754371 24520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:47.755548 24520 hybrid_clock.cc:648] HybridClock initialized: now 1781986367755532 us; error 38 us; skew 500 ppm
I20260620 20:12:47.757287 24520 webserver.cc:460] Webserver started at http://127.5.91.131:33843/ using document root <none> and password file <none>
I20260620 20:12:47.757822 24520 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:47.757884 24520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:47.758230 24520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:47.760246 24520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "bbfe985d65f544cfb6715678e1a704c0"
format_stamp: "Formatted at 2026-06-20 20:12:47 on dist-test-slave-l8h4"
server_key: "497259eaf7de094498a5541f85fb5621"
server_key_iv: "9a1d904e4a48fe96433d4611e0340436"
server_key_version: "encryptionkey@0"
I20260620 20:12:47.760766 24520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "bbfe985d65f544cfb6715678e1a704c0"
format_stamp: "Formatted at 2026-06-20 20:12:47 on dist-test-slave-l8h4"
server_key: "497259eaf7de094498a5541f85fb5621"
server_key_iv: "9a1d904e4a48fe96433d4611e0340436"
server_key_version: "encryptionkey@0"
I20260620 20:12:47.764276 24520 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 20:12:47.766571 24534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:47.767776 24520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:12:47.767884 24520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "bbfe985d65f544cfb6715678e1a704c0"
format_stamp: "Formatted at 2026-06-20 20:12:47 on dist-test-slave-l8h4"
server_key: "497259eaf7de094498a5541f85fb5621"
server_key_iv: "9a1d904e4a48fe96433d4611e0340436"
server_key_version: "encryptionkey@0"
I20260620 20:12:47.767979 24520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:47.782536 24520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:47.783185 24520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:47.783334 24520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:47.783844 24520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:47.784771 24520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:12:47.784819 24520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:47.784871 24520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:12:47.784901 24520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:47.793741 24520 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:44783
I20260620 20:12:47.793795 24647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:44783 every 8 connection(s)
I20260620 20:12:47.794642 24520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:12:47.799532 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 24520
I20260620 20:12:47.799660 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:12:47.799889 5486 external_mini_cluster.cc:1428] Setting key 635873c0ddf4236eb28f7e35afd17c0b
I20260620 20:12:47.803694 24237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:47.796484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:44543 (local address 127.5.91.190:33385)
0620 20:12:47.796673 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:47.796678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:47.797680 (+ 1002us) server_negotiation.cc:407] Connection header received
0620 20:12:47.799016 (+ 1336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:47.799018 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:47.799072 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:47.799151 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:47.800123 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.800782 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.801890 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.802111 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.802222 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:47.803194 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:47.803198 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:47.803200 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:12:47.803210 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:47.803232 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:12:47.803240 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:47.803305 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:47.803530 (+ 225us) server_negotiation.cc:299] Negotiation successful
0620 20:12:47.803584 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260620 20:12:47.804551 24650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:47.796861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:33385 (local address 127.5.91.131:44543)
0620 20:12:47.797551 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:47.797593 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:47.798814 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:47.799345 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:47.799354 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:47.799437 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:47.799958 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:47.799975 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.800885 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.800888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:47.801791 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:47.801801 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.802258 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.802261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:47.802398 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:47.802947 (+ 549us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 20:12:47.802948 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:47.802987 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 20:12:47.803056 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:47.803367 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:47.803376 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:47.803380 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 20:12:47.803490 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 20:12:47.803717 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":115,"threads_started":1}
I20260620 20:12:47.805555 24648 heartbeater.cc:347] Connected to a master server at 127.5.91.190:33385
I20260620 20:12:47.805778 24648 heartbeater.cc:464] Registering TS with master...
I20260620 20:12:47.806322 24648 heartbeater.cc:511] Master 127.5.91.190:33385 requested a full tablet report, sending...
I20260620 20:12:47.807313 24198 ts_manager.cc:194] Registered new tserver with Master: bbfe985d65f544cfb6715678e1a704c0 (127.5.91.131:44783)
I20260620 20:12:47.807878 24198 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.91.131:44543
I20260620 20:12:47.813427 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 20:12:47.818802 24237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:47.815297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58250 (local address 127.5.91.190:33385)
0620 20:12:47.815409 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:47.815412 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:47.815552 (+ 140us) server_negotiation.cc:407] Connection header received
0620 20:12:47.815629 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:47.815631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:47.815668 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:47.815730 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:47.815948 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.816434 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.817241 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.817466 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.817549 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:47.818350 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:47.818353 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:47.818355 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:12:47.818367 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:47.818385 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:12:47.818393 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:47.818491 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:47.818659 (+ 168us) server_negotiation.cc:299] Negotiation successful
0620 20:12:47.818697 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 20:12:47.821756 24198 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:12:47.824688 24198 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:12:47.838770 24660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:47.832941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:44783 (local address 127.0.0.1:57790)
0620 20:12:47.833813 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:47.833824 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:47.833893 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:47.834860 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:47.834863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:47.834877 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:47.834950 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:47.834954 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.835868 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.835872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:47.837311 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:47.837322 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.838394 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.838397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:47.838477 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:47.838497 (+ 20us) client_negotiation.cc:769] Sending connection context
0620 20:12:47.838543 (+ 46us) client_negotiation.cc:240] Negotiation successful
0620 20:12:47.838664 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"mutex_wait_us":74,"thread_start_us":89,"threads_started":1}
I20260620 20:12:47.838958 24663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:47.833536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57790 (local address 127.5.91.131:44783)
0620 20:12:47.834356 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:47.834367 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:47.834390 (+ 23us) server_negotiation.cc:407] Connection header received
0620 20:12:47.834461 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:47.834466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:47.834615 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:47.834739 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:47.835042 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.835767 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.837444 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.838260 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.838358 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:47.838520 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:47.838582 (+ 62us) server_negotiation.cc:299] Negotiation successful
0620 20:12:47.838785 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":149,"threads_started":1}
I20260620 20:12:47.839179 24661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:47.832974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58782 (local address 127.5.91.129:38427)
0620 20:12:47.833658 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:47.833670 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:47.833694 (+ 24us) server_negotiation.cc:407] Connection header received
0620 20:12:47.833917 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:47.833923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:47.834128 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:47.834285 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:47.834752 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.835949 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.837470 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.837995 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.838100 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:47.838322 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:47.838846 (+ 524us) server_negotiation.cc:299] Negotiation successful
0620 20:12:47.838952 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":136,"threads_started":1}
I20260620 20:12:47.839071 24662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:47.833162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:35585 (local address 127.0.0.1:34574)
0620 20:12:47.834083 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:47.834094 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:47.834178 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:47.834719 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:47.834722 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:47.834739 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:47.834834 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:47.834841 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.836115 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.836118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:47.837325 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:47.837335 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.838580 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.838584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:47.838771 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:47.838795 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 20:12:47.838880 (+ 85us) client_negotiation.cc:240] Negotiation successful
0620 20:12:47.838945 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":868,"thread_start_us":78,"threads_started":1}
I20260620 20:12:47.839395 24664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:47.833560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34574 (local address 127.5.91.130:35585)
0620 20:12:47.834101 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:47.834113 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:47.834179 (+ 66us) server_negotiation.cc:407] Connection header received
0620 20:12:47.834322 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:47.834327 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:47.834568 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:47.834727 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:47.834937 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.836003 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.837470 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.838260 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.838358 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:47.838520 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:47.839159 (+ 639us) server_negotiation.cc:299] Negotiation successful
0620 20:12:47.839252 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":123,"threads_started":1}
I20260620 20:12:47.838737 24659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:47.832830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:38427 (local address 127.0.0.1:58782)
0620 20:12:47.833554 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:47.833614 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:47.833733 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:47.834443 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:47.834471 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:47.834493 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:47.834632 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:47.834639 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.836056 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.836059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:47.837311 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:47.837322 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.838106 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.838110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:47.838253 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:47.838291 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 20:12:47.838390 (+ 99us) client_negotiation.cc:240] Negotiation successful
0620 20:12:47.838505 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"mutex_wait_us":25,"thread_start_us":71,"threads_started":1}
I20260620 20:12:47.842199 24451 tablet_service.cc:1467] Processing CreateTablet for tablet a1c45814f17348c0b61eb979f4a81e64 (DEFAULT_TABLE table=test-table [id=fb57d7a6b1194f09906188ef1bf3671c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:12:47.842206 24320 tablet_service.cc:1467] Processing CreateTablet for tablet a1c45814f17348c0b61eb979f4a81e64 (DEFAULT_TABLE table=test-table [id=fb57d7a6b1194f09906188ef1bf3671c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:12:47.841929 24582 tablet_service.cc:1467] Processing CreateTablet for tablet a1c45814f17348c0b61eb979f4a81e64 (DEFAULT_TABLE table=test-table [id=fb57d7a6b1194f09906188ef1bf3671c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:12:47.843318 24451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1c45814f17348c0b61eb979f4a81e64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:47.843318 24320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1c45814f17348c0b61eb979f4a81e64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:47.844138 24582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1c45814f17348c0b61eb979f4a81e64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:47.850231 24665 tablet_bootstrap.cc:492] T a1c45814f17348c0b61eb979f4a81e64 P bbfe985d65f544cfb6715678e1a704c0: Bootstrap starting.
I20260620 20:12:47.851410 24667 tablet_bootstrap.cc:492] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e: Bootstrap starting.
I20260620 20:12:47.851410 24666 tablet_bootstrap.cc:492] T a1c45814f17348c0b61eb979f4a81e64 P ca61bf94299b4bbf84ca4da1fcb7c616: Bootstrap starting.
I20260620 20:12:47.852311 24665 tablet_bootstrap.cc:654] T a1c45814f17348c0b61eb979f4a81e64 P bbfe985d65f544cfb6715678e1a704c0: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:47.853149 24665 log.cc:824] T a1c45814f17348c0b61eb979f4a81e64 P bbfe985d65f544cfb6715678e1a704c0: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:47.853417 24667 tablet_bootstrap.cc:654] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:47.853417 24666 tablet_bootstrap.cc:654] T a1c45814f17348c0b61eb979f4a81e64 P ca61bf94299b4bbf84ca4da1fcb7c616: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:47.854346 24666 log.cc:824] T a1c45814f17348c0b61eb979f4a81e64 P ca61bf94299b4bbf84ca4da1fcb7c616: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:47.854346 24667 log.cc:824] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:47.855003 24665 tablet_bootstrap.cc:492] T a1c45814f17348c0b61eb979f4a81e64 P bbfe985d65f544cfb6715678e1a704c0: No bootstrap required, opened a new log
I20260620 20:12:47.855198 24665 ts_tablet_manager.cc:1397] T a1c45814f17348c0b61eb979f4a81e64 P bbfe985d65f544cfb6715678e1a704c0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 20:12:47.856199 24666 tablet_bootstrap.cc:492] T a1c45814f17348c0b61eb979f4a81e64 P ca61bf94299b4bbf84ca4da1fcb7c616: No bootstrap required, opened a new log
I20260620 20:12:47.856388 24666 ts_tablet_manager.cc:1397] T a1c45814f17348c0b61eb979f4a81e64 P ca61bf94299b4bbf84ca4da1fcb7c616: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 20:12:47.856665 24667 tablet_bootstrap.cc:492] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e: No bootstrap required, opened a new log
I20260620 20:12:47.856848 24667 ts_tablet_manager.cc:1397] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 20:12:47.858592 24665 raft_consensus.cc:348] T a1c45814f17348c0b61eb979f4a81e64 P bbfe985d65f544cfb6715678e1a704c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca61bf94299b4bbf84ca4da1fcb7c616" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38427 } } peers { permanent_uuid: "bbfe985d65f544cfb6715678e1a704c0" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 44783 } } peers { permanent_uuid: "9572de4f62cf4a7aacabf6bfe5d5812e" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 35585 } }
I20260620 20:12:47.858850 24665 raft_consensus.cc:374] T a1c45814f17348c0b61eb979f4a81e64 P bbfe985d65f544cfb6715678e1a704c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:47.858898 24665 raft_consensus.cc:729] T a1c45814f17348c0b61eb979f4a81e64 P bbfe985d65f544cfb6715678e1a704c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbfe985d65f544cfb6715678e1a704c0, State: Initialized, Role: FOLLOWER
I20260620 20:12:47.859380 24665 consensus_queue.cc:260] T a1c45814f17348c0b61eb979f4a81e64 P bbfe985d65f544cfb6715678e1a704c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca61bf94299b4bbf84ca4da1fcb7c616" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38427 } } peers { permanent_uuid: "bbfe985d65f544cfb6715678e1a704c0" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 44783 } } peers { permanent_uuid: "9572de4f62cf4a7aacabf6bfe5d5812e" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 35585 } }
I20260620 20:12:47.859957 24648 heartbeater.cc:503] Master 127.5.91.190:33385 was elected leader, sending a full tablet report...
I20260620 20:12:47.860311 24665 ts_tablet_manager.cc:1428] T a1c45814f17348c0b61eb979f4a81e64 P bbfe985d65f544cfb6715678e1a704c0: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260620 20:12:47.860424 24666 raft_consensus.cc:348] T a1c45814f17348c0b61eb979f4a81e64 P ca61bf94299b4bbf84ca4da1fcb7c616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca61bf94299b4bbf84ca4da1fcb7c616" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38427 } } peers { permanent_uuid: "bbfe985d65f544cfb6715678e1a704c0" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 44783 } } peers { permanent_uuid: "9572de4f62cf4a7aacabf6bfe5d5812e" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 35585 } }
I20260620 20:12:47.860424 24667 raft_consensus.cc:348] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca61bf94299b4bbf84ca4da1fcb7c616" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38427 } } peers { permanent_uuid: "bbfe985d65f544cfb6715678e1a704c0" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 44783 } } peers { permanent_uuid: "9572de4f62cf4a7aacabf6bfe5d5812e" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 35585 } }
I20260620 20:12:47.860679 24667 raft_consensus.cc:374] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:47.860679 24666 raft_consensus.cc:374] T a1c45814f17348c0b61eb979f4a81e64 P ca61bf94299b4bbf84ca4da1fcb7c616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:47.860728 24667 raft_consensus.cc:729] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9572de4f62cf4a7aacabf6bfe5d5812e, State: Initialized, Role: FOLLOWER
I20260620 20:12:47.860728 24666 raft_consensus.cc:729] T a1c45814f17348c0b61eb979f4a81e64 P ca61bf94299b4bbf84ca4da1fcb7c616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca61bf94299b4bbf84ca4da1fcb7c616, State: Initialized, Role: FOLLOWER
I20260620 20:12:47.861166 24666 consensus_queue.cc:260] T a1c45814f17348c0b61eb979f4a81e64 P ca61bf94299b4bbf84ca4da1fcb7c616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca61bf94299b4bbf84ca4da1fcb7c616" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38427 } } peers { permanent_uuid: "bbfe985d65f544cfb6715678e1a704c0" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 44783 } } peers { permanent_uuid: "9572de4f62cf4a7aacabf6bfe5d5812e" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 35585 } }
I20260620 20:12:47.861166 24667 consensus_queue.cc:260] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca61bf94299b4bbf84ca4da1fcb7c616" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38427 } } peers { permanent_uuid: "bbfe985d65f544cfb6715678e1a704c0" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 44783 } } peers { permanent_uuid: "9572de4f62cf4a7aacabf6bfe5d5812e" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 35585 } }
I20260620 20:12:47.861985 24517 heartbeater.cc:503] Master 127.5.91.190:33385 was elected leader, sending a full tablet report...
I20260620 20:12:47.862071 24667 ts_tablet_manager.cc:1428] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260620 20:12:47.862078 24666 ts_tablet_manager.cc:1428] T a1c45814f17348c0b61eb979f4a81e64 P ca61bf94299b4bbf84ca4da1fcb7c616: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260620 20:12:47.862406 24386 heartbeater.cc:503] Master 127.5.91.190:33385 was elected leader, sending a full tablet report...
W20260620 20:12:47.870306 24518 tablet.cc:2369] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:12:47.889405 24672 raft_consensus.cc:482] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:12:47.889597 24672 raft_consensus.cc:504] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca61bf94299b4bbf84ca4da1fcb7c616" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38427 } } peers { permanent_uuid: "bbfe985d65f544cfb6715678e1a704c0" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 44783 } } peers { permanent_uuid: "9572de4f62cf4a7aacabf6bfe5d5812e" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 35585 } }
I20260620 20:12:47.890781 24672 leader_election.cc:288] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca61bf94299b4bbf84ca4da1fcb7c616 (127.5.91.129:38427), bbfe985d65f544cfb6715678e1a704c0 (127.5.91.131:44783)
I20260620 20:12:47.894244 24519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:47.891028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:44783 (local address 127.5.91.130:48861)
0620 20:12:47.891200 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:47.891214 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:47.891302 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:47.891550 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:47.891553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:47.891569 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:47.891675 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:47.891680 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.892399 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.892402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:47.893222 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:47.893231 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.893826 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.893829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:47.893925 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:47.893947 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 20:12:47.894014 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 20:12:47.894085 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":3}
I20260620 20:12:47.894244 24663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:47.891123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:48861 (local address 127.5.91.131:44783)
0620 20:12:47.891238 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:47.891243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:47.891256 (+ 13us) server_negotiation.cc:407] Connection header received
0620 20:12:47.891410 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:47.891412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:47.891456 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:47.891525 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:47.891771 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.892265 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.893342 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.893719 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.893821 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:47.893895 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:47.894038 (+ 143us) server_negotiation.cc:299] Negotiation successful
0620 20:12:47.894101 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 20:12:47.894754 24661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:47.891112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:52921 (local address 127.5.91.129:38427)
0620 20:12:47.891233 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:47.891238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:47.891490 (+ 252us) server_negotiation.cc:407] Connection header received
0620 20:12:47.891631 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:47.891634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:47.891680 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:47.891738 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:47.891990 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.892573 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.893587 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.893982 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.894079 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:47.894125 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:47.894466 (+ 341us) server_negotiation.cc:299] Negotiation successful
0620 20:12:47.894549 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 20:12:47.895037 24602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1c45814f17348c0b61eb979f4a81e64" candidate_uuid: "9572de4f62cf4a7aacabf6bfe5d5812e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbfe985d65f544cfb6715678e1a704c0" is_pre_election: true
I20260620 20:12:47.895315 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:47.891052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:38427 (local address 127.5.91.130:52921)
0620 20:12:47.891395 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:47.891406 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:47.891532 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:47.891759 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:47.891761 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:47.891775 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:47.891884 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:47.891889 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.892671 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.892675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:47.893470 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:47.893479 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.894080 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.894083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:47.894230 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:47.894246 (+ 16us) client_negotiation.cc:769] Sending connection context
0620 20:12:47.895038 (+ 792us) client_negotiation.cc:240] Negotiation successful
0620 20:12:47.895120 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":30,"thread_start_us":126,"threads_started":1}
I20260620 20:12:47.895391 24602 raft_consensus.cc:2454] T a1c45814f17348c0b61eb979f4a81e64 P bbfe985d65f544cfb6715678e1a704c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9572de4f62cf4a7aacabf6bfe5d5812e in term 0.
I20260620 20:12:47.895881 24407 leader_election.cc:302] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9572de4f62cf4a7aacabf6bfe5d5812e, bbfe985d65f544cfb6715678e1a704c0; no voters:
I20260620 20:12:47.895982 24340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1c45814f17348c0b61eb979f4a81e64" candidate_uuid: "9572de4f62cf4a7aacabf6bfe5d5812e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca61bf94299b4bbf84ca4da1fcb7c616" is_pre_election: true
I20260620 20:12:47.896173 24672 raft_consensus.cc:2781] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:12:47.896257 24672 raft_consensus.cc:482] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:12:47.896250 24340 raft_consensus.cc:2454] T a1c45814f17348c0b61eb979f4a81e64 P ca61bf94299b4bbf84ca4da1fcb7c616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9572de4f62cf4a7aacabf6bfe5d5812e in term 0.
I20260620 20:12:47.896309 24672 raft_consensus.cc:3037] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:47.897330 24672 raft_consensus.cc:504] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca61bf94299b4bbf84ca4da1fcb7c616" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38427 } } peers { permanent_uuid: "bbfe985d65f544cfb6715678e1a704c0" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 44783 } } peers { permanent_uuid: "9572de4f62cf4a7aacabf6bfe5d5812e" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 35585 } }
I20260620 20:12:47.897668 24672 leader_election.cc:288] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e [CANDIDATE]: Term 1 election: Requested vote from peers ca61bf94299b4bbf84ca4da1fcb7c616 (127.5.91.129:38427), bbfe985d65f544cfb6715678e1a704c0 (127.5.91.131:44783)
I20260620 20:12:47.898011 24340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1c45814f17348c0b61eb979f4a81e64" candidate_uuid: "9572de4f62cf4a7aacabf6bfe5d5812e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca61bf94299b4bbf84ca4da1fcb7c616"
I20260620 20:12:47.898160 24340 raft_consensus.cc:3037] T a1c45814f17348c0b61eb979f4a81e64 P ca61bf94299b4bbf84ca4da1fcb7c616 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:47.898240 24602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1c45814f17348c0b61eb979f4a81e64" candidate_uuid: "9572de4f62cf4a7aacabf6bfe5d5812e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbfe985d65f544cfb6715678e1a704c0"
I20260620 20:12:47.898366 24602 raft_consensus.cc:3037] T a1c45814f17348c0b61eb979f4a81e64 P bbfe985d65f544cfb6715678e1a704c0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:47.899267 24340 raft_consensus.cc:2454] T a1c45814f17348c0b61eb979f4a81e64 P ca61bf94299b4bbf84ca4da1fcb7c616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9572de4f62cf4a7aacabf6bfe5d5812e in term 1.
I20260620 20:12:47.899544 24406 leader_election.cc:302] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9572de4f62cf4a7aacabf6bfe5d5812e, ca61bf94299b4bbf84ca4da1fcb7c616; no voters:
I20260620 20:12:47.899701 24672 raft_consensus.cc:2781] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:12:47.899899 24672 raft_consensus.cc:686] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e [term 1 LEADER]: Becoming Leader. State: Replica: 9572de4f62cf4a7aacabf6bfe5d5812e, State: Running, Role: LEADER
I20260620 20:12:47.899925 24602 raft_consensus.cc:2454] T a1c45814f17348c0b61eb979f4a81e64 P bbfe985d65f544cfb6715678e1a704c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9572de4f62cf4a7aacabf6bfe5d5812e in term 1.
I20260620 20:12:47.900215 24672 consensus_queue.cc:237] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca61bf94299b4bbf84ca4da1fcb7c616" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38427 } } peers { permanent_uuid: "bbfe985d65f544cfb6715678e1a704c0" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 44783 } } peers { permanent_uuid: "9572de4f62cf4a7aacabf6bfe5d5812e" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 35585 } }
I20260620 20:12:47.903149 24197 catalog_manager.cc:5495] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9572de4f62cf4a7aacabf6bfe5d5812e (127.5.91.130). New cstate: current_term: 1 leader_uuid: "9572de4f62cf4a7aacabf6bfe5d5812e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca61bf94299b4bbf84ca4da1fcb7c616" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbfe985d65f544cfb6715678e1a704c0" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 44783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9572de4f62cf4a7aacabf6bfe5d5812e" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 35585 } health_report { overall_health: HEALTHY } } }
I20260620 20:12:47.922819 24664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:47.919295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34582 (local address 127.5.91.130:35585)
0620 20:12:47.919445 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:47.919448 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:47.919463 (+ 15us) server_negotiation.cc:407] Connection header received
0620 20:12:47.919541 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:47.919543 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:47.919596 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:47.919686 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:12:47.919921 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.920405 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.921255 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:47.921489 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:47.921576 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:47.921895 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:12:47.922362 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:12:47.922522 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:47.922618 (+ 96us) server_negotiation.cc:299] Negotiation successful
0620 20:12:47.922667 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260620 20:12:47.928354 24340 raft_consensus.cc:1261] T a1c45814f17348c0b61eb979f4a81e64 P ca61bf94299b4bbf84ca4da1fcb7c616 [term 1 FOLLOWER]: Refusing update from remote peer 9572de4f62cf4a7aacabf6bfe5d5812e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:12:47.928352 24602 raft_consensus.cc:1261] T a1c45814f17348c0b61eb979f4a81e64 P bbfe985d65f544cfb6715678e1a704c0 [term 1 FOLLOWER]: Refusing update from remote peer 9572de4f62cf4a7aacabf6bfe5d5812e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:12:47.929059 24672 consensus_queue.cc:1035] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca61bf94299b4bbf84ca4da1fcb7c616" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 38427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:12:47.929241 24675 consensus_queue.cc:1035] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbfe985d65f544cfb6715678e1a704c0" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 44783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:12:47.935632 24678 mvcc.cc:205] Tried to move back new op lower bound from 7299016163027980288 to 7299016162921775104. Current Snapshot: MvccSnapshot[applied={T|T < 7299016163027980288}]
I20260620 20:12:47.936385 24680 mvcc.cc:205] Tried to move back new op lower bound from 7299016163027980288 to 7299016162921775104. Current Snapshot: MvccSnapshot[applied={T|T < 7299016163027980288}]
I20260620 20:12:47.943622 24198 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:12:47.943759 24198 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:12:47.946213 24198 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b3ba43b66794b3abab8dcaa9007d68f: Sending DeleteTablet for 3 replicas of tablet a1c45814f17348c0b61eb979f4a81e64
I20260620 20:12:47.947078 24320 tablet_service.cc:1514] Processing DeleteTablet for tablet a1c45814f17348c0b61eb979f4a81e64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:12:47 UTC) from {username='slave'} at 127.0.0.1:58782
I20260620 20:12:47.947103 24582 tablet_service.cc:1514] Processing DeleteTablet for tablet a1c45814f17348c0b61eb979f4a81e64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:12:47 UTC) from {username='slave'} at 127.0.0.1:57790
I20260620 20:12:47.947336 24451 tablet_service.cc:1514] Processing DeleteTablet for tablet a1c45814f17348c0b61eb979f4a81e64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:12:47 UTC) from {username='slave'} at 127.0.0.1:34574
I20260620 20:12:47.947767 24689 tablet_replica.cc:330] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e: stopping tablet replica
I20260620 20:12:47.947973 24687 tablet_replica.cc:330] T a1c45814f17348c0b61eb979f4a81e64 P ca61bf94299b4bbf84ca4da1fcb7c616: stopping tablet replica
I20260620 20:12:47.948062 24689 raft_consensus.cc:2229] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e [term 1 LEADER]: Raft consensus shutting down.
I20260620 20:12:47.948076 24688 tablet_replica.cc:330] T a1c45814f17348c0b61eb979f4a81e64 P bbfe985d65f544cfb6715678e1a704c0: stopping tablet replica
I20260620 20:12:47.948223 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 24258
I20260620 20:12:47.948446 24689 raft_consensus.cc:2258] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:12:47.950452 24688 raft_consensus.cc:2229] T a1c45814f17348c0b61eb979f4a81e64 P bbfe985d65f544cfb6715678e1a704c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:12:47.950551 24689 ts_tablet_manager.cc:1905] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:12:47.950882 24688 raft_consensus.cc:2258] T a1c45814f17348c0b61eb979f4a81e64 P bbfe985d65f544cfb6715678e1a704c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:12:47.952168 24688 ts_tablet_manager.cc:1905] T a1c45814f17348c0b61eb979f4a81e64 P bbfe985d65f544cfb6715678e1a704c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:12:47.953603 24689 ts_tablet_manager.cc:1918] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:12:47.953696 24689 log.cc:1196] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-1/wal/wals/a1c45814f17348c0b61eb979f4a81e64
I20260620 20:12:47.953981 24689 ts_tablet_manager.cc:1939] T a1c45814f17348c0b61eb979f4a81e64 P 9572de4f62cf4a7aacabf6bfe5d5812e: Deleting consensus metadata
I20260620 20:12:47.954939 24688 ts_tablet_manager.cc:1918] T a1c45814f17348c0b61eb979f4a81e64 P bbfe985d65f544cfb6715678e1a704c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:12:47.955031 24688 log.cc:1196] T a1c45814f17348c0b61eb979f4a81e64 P bbfe985d65f544cfb6715678e1a704c0: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/a1c45814f17348c0b61eb979f4a81e64
I20260620 20:12:47.955214 24182 catalog_manager.cc:4841] TS 9572de4f62cf4a7aacabf6bfe5d5812e (127.5.91.130:35585): tablet a1c45814f17348c0b61eb979f4a81e64 (table test-table [id=fb57d7a6b1194f09906188ef1bf3671c]) successfully deleted
I20260620 20:12:47.955313 24688 ts_tablet_manager.cc:1939] T a1c45814f17348c0b61eb979f4a81e64 P bbfe985d65f544cfb6715678e1a704c0: Deleting consensus metadata
I20260620 20:12:47.956416 24185 catalog_manager.cc:4841] TS bbfe985d65f544cfb6715678e1a704c0 (127.5.91.131:44783): tablet a1c45814f17348c0b61eb979f4a81e64 (table test-table [id=fb57d7a6b1194f09906188ef1bf3671c]) successfully deleted
W20260620 20:12:47.956823 24184 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.91.129:38427 (error 108)
I20260620 20:12:47.956966 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 24389
I20260620 20:12:47.958524 24659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:47.958240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:38427 (local address 127.0.0.1:58796)
0620 20:12:47.958383 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:47.958439 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.129:38427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260620 20:12:47.958666 24184 catalog_manager.cc:4584] TS ca61bf94299b4bbf84ca4da1fcb7c616 (127.5.91.129:38427): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a1c45814f17348c0b61eb979f4a81e64: Network error: Client connection negotiation failed: client connection to 127.5.91.129:38427: connect: Connection refused (error 111)
I20260620 20:12:47.961879 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 24520
I20260620 20:12:47.966114 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 24167
2026-06-20T20:12:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (999 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:12:48 dist-test-slave-l8h4 krb5kdc[24693](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:12:48 dist-test-slave-l8h4 krb5kdc[24693](info): set up 2 sockets
Jun 20 20:12:48 dist-test-slave-l8h4 krb5kdc[24693](info): commencing operation
krb5kdc: starting...
W20260620 20:12:48.934689 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.949s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:12:49 dist-test-slave-l8h4 krb5kdc[24693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:12:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:12:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:12:49.055190 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:46067
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:37167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:46067
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:12:49.161124 24706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:49.161397 24706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:49.161456 24706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:49.165740 24706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:12:49.165812 24706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:49.165838 24706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:49.165859 24706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:12:49.165880 24706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:12:49.171218 24706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:37167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:46067
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:46067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:49.172217 24706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:49.173159 24706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:49.181273 24711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:49.181365 24706 server_base.cc:1034] running on GCE node
W20260620 20:12:49.181273 24714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:49.181273 24712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:49.182099 24706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:49.183162 24706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:49.184348 24706 hybrid_clock.cc:648] HybridClock initialized: now 1781986369184325 us; error 36 us; skew 500 ppm
Jun 20 20:12:49 dist-test-slave-l8h4 krb5kdc[24693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986369, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:49.187243 24706 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:12:49.188102 24706 webserver.cc:460] Webserver started at http://127.5.91.190:40567/ using document root <none> and password file <none>
I20260620 20:12:49.188624 24706 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:49.188689 24706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:49.188982 24706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:49.191075 24706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "e8638b92dce9409fa1a2c8ab9a8daa5c"
format_stamp: "Formatted at 2026-06-20 20:12:49 on dist-test-slave-l8h4"
server_key: "54129dd6f3871de98707681e38ef869b"
server_key_iv: "1352e854d37fe9b25b0c482e4da352f4"
server_key_version: "encryptionkey@0"
I20260620 20:12:49.191576 24706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "e8638b92dce9409fa1a2c8ab9a8daa5c"
format_stamp: "Formatted at 2026-06-20 20:12:49 on dist-test-slave-l8h4"
server_key: "54129dd6f3871de98707681e38ef869b"
server_key_iv: "1352e854d37fe9b25b0c482e4da352f4"
server_key_version: "encryptionkey@0"
I20260620 20:12:49.195127 24706 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260620 20:12:49.197633 24721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:49.198804 24706 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 20:12:49.198918 24706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "e8638b92dce9409fa1a2c8ab9a8daa5c"
format_stamp: "Formatted at 2026-06-20 20:12:49 on dist-test-slave-l8h4"
server_key: "54129dd6f3871de98707681e38ef869b"
server_key_iv: "1352e854d37fe9b25b0c482e4da352f4"
server_key_version: "encryptionkey@0"
I20260620 20:12:49.199015 24706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:49.217819 24706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:49.220556 24706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:49.220760 24706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:49.227735 24706 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:46067
I20260620 20:12:49.227789 24773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:46067 every 8 connection(s)
I20260620 20:12:49.228683 24706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:12:49.231355 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 24706
I20260620 20:12:49.231491 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:12:49.231798 5486 external_mini_cluster.cc:1428] Setting key 7e38b7fcd9ad37c3ad2d423412c5acb1
I20260620 20:12:49.231937 24774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:49.238219 24774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c: Bootstrap starting.
Jun 20 20:12:49 dist-test-slave-l8h4 krb5kdc[24693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:12:49.240581 24774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:49.241272 24774 log.cc:824] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:49.243168 24774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c: No bootstrap required, opened a new log
I20260620 20:12:49.245854 24777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:49.233304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49910 (local address 127.5.91.190:46067)
0620 20:12:49.233742 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:49.233756 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:49.233790 (+ 34us) server_negotiation.cc:407] Connection header received
0620 20:12:49.234658 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:49.234676 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:49.235010 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:49.235311 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:49.236067 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.237046 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.237846 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.238361 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.238526 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:49.241163 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:49.241173 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:49.241186 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:49.241217 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:49.242926 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:49.243326 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:49.243331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:49.243337 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:49.243472 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:49.243712 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:49.243715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:49.243716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:49.244194 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:49.244361 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:49.244604 (+ 243us) server_negotiation.cc:299] Negotiation successful
0620 20:12:49.244891 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":136,"threads_started":1}
I20260620 20:12:49.247298 24774 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8638b92dce9409fa1a2c8ab9a8daa5c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46067 } }
I20260620 20:12:49.247771 24774 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:49.247949 24774 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8638b92dce9409fa1a2c8ab9a8daa5c, State: Initialized, Role: FOLLOWER
I20260620 20:12:49.248625 24774 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8638b92dce9409fa1a2c8ab9a8daa5c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46067 } }
I20260620 20:12:49.248906 24774 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:12:49.248996 24774 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:12:49.249111 24774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:49.250686 24774 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8638b92dce9409fa1a2c8ab9a8daa5c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46067 } }
I20260620 20:12:49.251082 24774 leader_election.cc:302] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8638b92dce9409fa1a2c8ab9a8daa5c; no voters:
I20260620 20:12:49.251425 24774 leader_election.cc:288] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:12:49.251565 24779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:12:49.251773 24779 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c [term 1 LEADER]: Becoming Leader. State: Replica: e8638b92dce9409fa1a2c8ab9a8daa5c, State: Running, Role: LEADER
I20260620 20:12:49.252146 24779 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8638b92dce9409fa1a2c8ab9a8daa5c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46067 } }
I20260620 20:12:49.252975 24774 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:12:49.253950 24780 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8638b92dce9409fa1a2c8ab9a8daa5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8638b92dce9409fa1a2c8ab9a8daa5c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46067 } } }
I20260620 20:12:49.253950 24781 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8638b92dce9409fa1a2c8ab9a8daa5c. Latest consensus state: current_term: 1 leader_uuid: "e8638b92dce9409fa1a2c8ab9a8daa5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8638b92dce9409fa1a2c8ab9a8daa5c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46067 } } }
I20260620 20:12:49.254103 24780 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:49.254103 24781 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:49.254623 24788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:12:49.257381 24788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:12:49.264289 24788 catalog_manager.cc:1347] Generated new cluster ID: 2faf86f651d84990a482104c3f829e8c
I20260620 20:12:49.264360 24788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:12:49.273209 24788 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:12:49.274588 24788 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:12:49.282773 24788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c: Generated new TSK 0
I20260620 20:12:49.283732 24788 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.91.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
Entry for principal kudu/127.5.91.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab.
I20260620 20:12:49.317844 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46067
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:37167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:12:49.425756 24800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:49.425997 24800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:49.426051 24800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:49.429972 24800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:49.430028 24800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:49.430109 24800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:12:49.435182 24800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:37167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.129.keytab
--principal=kudu/127.5.91.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:49.436146 24800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:49.437104 24800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:49.446703 24808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:49.446767 24800 server_base.cc:1034] running on GCE node
W20260620 20:12:49.446714 24805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:49.446714 24806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:49.447387 24800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:49.448012 24800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:49.449186 24800 hybrid_clock.cc:648] HybridClock initialized: now 1781986369449165 us; error 33 us; skew 500 ppm
Jun 20 20:12:49 dist-test-slave-l8h4 krb5kdc[24693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986369, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:49.451993 24800 init.cc:377] Logged in from keytab as kudu/127.5.91.129@KRBTEST.COM (short username kudu)
I20260620 20:12:49.452888 24800 webserver.cc:460] Webserver started at http://127.5.91.129:45631/ using document root <none> and password file <none>
I20260620 20:12:49.453441 24800 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:49.453505 24800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:49.453691 24800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:49.455783 24800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "0b378aedf5c14370a10cf6757b09f8d5"
format_stamp: "Formatted at 2026-06-20 20:12:49 on dist-test-slave-l8h4"
server_key: "ecc205b321de21238aae05cc22afa1c1"
server_key_iv: "c3fc13d6d8d7f5e272088b94218a6de8"
server_key_version: "encryptionkey@0"
I20260620 20:12:49.456306 24800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "0b378aedf5c14370a10cf6757b09f8d5"
format_stamp: "Formatted at 2026-06-20 20:12:49 on dist-test-slave-l8h4"
server_key: "ecc205b321de21238aae05cc22afa1c1"
server_key_iv: "c3fc13d6d8d7f5e272088b94218a6de8"
server_key_version: "encryptionkey@0"
I20260620 20:12:49.459898 24800 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 20:12:49.462491 24815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:49.463735 24800 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:12:49.463866 24800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "0b378aedf5c14370a10cf6757b09f8d5"
format_stamp: "Formatted at 2026-06-20 20:12:49 on dist-test-slave-l8h4"
server_key: "ecc205b321de21238aae05cc22afa1c1"
server_key_iv: "c3fc13d6d8d7f5e272088b94218a6de8"
server_key_version: "encryptionkey@0"
I20260620 20:12:49.463979 24800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:49.480911 24800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:49.483989 24800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:49.484179 24800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:49.484746 24800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:49.485584 24800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:12:49.485633 24800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:49.485685 24800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:12:49.485714 24800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:49.494441 24800 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:37349
I20260620 20:12:49.494472 24928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:37349 every 8 connection(s)
I20260620 20:12:49.495280 24800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
Jun 20 20:12:49 dist-test-slave-l8h4 krb5kdc[24693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986369, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.129@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:12:49.504385 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 24800
I20260620 20:12:49.504500 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:12:49.504822 5486 external_mini_cluster.cc:1428] Setting key c6e82f990bf40b09a0842fe608858beb
I20260620 20:12:49.508198 24777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:49.496928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:57139 (local address 127.5.91.190:46067)
0620 20:12:49.497064 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:49.497069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:49.497893 (+ 824us) server_negotiation.cc:407] Connection header received
0620 20:12:49.498738 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:49.498740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:49.498786 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:49.498854 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:49.499934 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.500475 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.501679 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.501928 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.502030 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:49.504623 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:49.504627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:49.504630 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:49.504661 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:49.506165 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:49.506649 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:49.506653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:49.506655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:49.506753 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:49.507103 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:49.507106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:49.507108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:49.507327 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:49.507431 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:49.507915 (+ 484us) server_negotiation.cc:299] Negotiation successful
0620 20:12:49.508041 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 20:12:49.509016 24931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:49.497307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46067 (local address 127.5.91.129:57139)
0620 20:12:49.497747 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:49.497786 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:49.498563 (+ 777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:49.498992 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:49.498997 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:49.499332 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:49.499796 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:49.499807 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.500627 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.500631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:49.501550 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:49.501560 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.502093 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.502098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:49.502296 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:49.502824 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:12:49.502841 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:49.504453 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:49.506271 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:49.506277 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:49.506291 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:49.506547 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:49.506849 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:49.506852 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:49.506853 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:49.507016 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:49.507450 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:49.507454 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:49.507682 (+ 228us) client_negotiation.cc:769] Sending connection context
0620 20:12:49.507856 (+ 174us) client_negotiation.cc:240] Negotiation successful
0620 20:12:49.508197 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":107,"threads_started":1}
I20260620 20:12:49.510331 24929 heartbeater.cc:347] Connected to a master server at 127.5.91.190:46067
I20260620 20:12:49.510586 24929 heartbeater.cc:464] Registering TS with master...
I20260620 20:12:49.511157 24929 heartbeater.cc:511] Master 127.5.91.190:46067 requested a full tablet report, sending...
I20260620 20:12:49.512792 24738 ts_manager.cc:194] Registered new tserver with Master: 0b378aedf5c14370a10cf6757b09f8d5 (127.5.91.129:37349)
WARNING: no policy specified for kudu/127.5.91.130@KRBTEST.COM; defaulting to no policy
I20260620 20:12:49.514348 24738 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.129@KRBTEST.COM'} at 127.5.91.129:57139
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
Entry for principal kudu/127.5.91.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab.
I20260620 20:12:49.531086 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46067
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:37167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:12:49.634598 24934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:49.634824 24934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:49.634863 24934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:49.638792 24934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:49.638846 24934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:49.638902 24934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:12:49.644783 24934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:37167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.130.keytab
--principal=kudu/127.5.91.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:49.645763 24934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:49.646752 24934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:49.656297 24939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:49.656301 24940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:49.656291 24942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:49.656590 24934 server_base.cc:1034] running on GCE node
I20260620 20:12:49.657196 24934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:49.657866 24934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:49.659047 24934 hybrid_clock.cc:648] HybridClock initialized: now 1781986369659019 us; error 39 us; skew 500 ppm
Jun 20 20:12:49 dist-test-slave-l8h4 krb5kdc[24693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986369, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:49.661649 24934 init.cc:377] Logged in from keytab as kudu/127.5.91.130@KRBTEST.COM (short username kudu)
I20260620 20:12:49.662489 24934 webserver.cc:460] Webserver started at http://127.5.91.130:37781/ using document root <none> and password file <none>
I20260620 20:12:49.662992 24934 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:49.663044 24934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:49.663197 24934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:49.665151 24934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "9011b1b2d4d64262b7b7876c2ed6bc2f"
format_stamp: "Formatted at 2026-06-20 20:12:49 on dist-test-slave-l8h4"
server_key: "c850f9b3e737e2acb4707d28bca69885"
server_key_iv: "82172b68c25691897beb42c426959ff4"
server_key_version: "encryptionkey@0"
I20260620 20:12:49.665634 24934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "9011b1b2d4d64262b7b7876c2ed6bc2f"
format_stamp: "Formatted at 2026-06-20 20:12:49 on dist-test-slave-l8h4"
server_key: "c850f9b3e737e2acb4707d28bca69885"
server_key_iv: "82172b68c25691897beb42c426959ff4"
server_key_version: "encryptionkey@0"
I20260620 20:12:49.669198 24934 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 20:12:49.671731 24949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:49.673224 24934 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 20:12:49.673339 24934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "9011b1b2d4d64262b7b7876c2ed6bc2f"
format_stamp: "Formatted at 2026-06-20 20:12:49 on dist-test-slave-l8h4"
server_key: "c850f9b3e737e2acb4707d28bca69885"
server_key_iv: "82172b68c25691897beb42c426959ff4"
server_key_version: "encryptionkey@0"
I20260620 20:12:49.673456 24934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:49.683784 24934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:49.686806 24934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:49.686985 24934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:49.687539 24934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:49.688341 24934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:12:49.688400 24934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:49.688444 24934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:12:49.688470 24934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:49.697258 24934 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:33129
I20260620 20:12:49.697274 25062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:33129 every 8 connection(s)
I20260620 20:12:49.698122 24934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:12:49.706806 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 24934
I20260620 20:12:49.706983 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
Jun 20 20:12:49 dist-test-slave-l8h4 krb5kdc[24693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986369, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.130@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:12:49.707278 5486 external_mini_cluster.cc:1428] Setting key e27ad399cd1dc8869e5a5702968cb2af
I20260620 20:12:49.711395 24777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:49.699873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:58167 (local address 127.5.91.190:46067)
0620 20:12:49.700014 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:49.700017 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:49.700785 (+ 768us) server_negotiation.cc:407] Connection header received
0620 20:12:49.702030 (+ 1245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:49.702033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:49.702088 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:49.702177 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:49.703282 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.703804 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.704896 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.705118 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.705201 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:49.707685 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:49.707688 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:49.707690 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:49.707712 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:49.709201 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:49.709747 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:49.709750 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:49.709751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:49.709837 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:49.710261 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:49.710265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:49.710267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:49.710514 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:49.710628 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:49.711107 (+ 479us) server_negotiation.cc:299] Negotiation successful
0620 20:12:49.711234 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 20:12:49.712105 25065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:49.700197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46067 (local address 127.5.91.130:58167)
0620 20:12:49.700653 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:49.700691 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:49.701826 (+ 1135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:49.702327 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:49.702333 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:49.702675 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:49.703138 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:49.703151 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.703918 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.703921 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:49.704777 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:49.704787 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.705249 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.705253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:49.705401 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:49.705939 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:12:49.705955 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:49.707517 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:49.709315 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:49.709321 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:49.709334 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:49.709643 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:49.709927 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:49.709930 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:49.709932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:49.710105 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:49.710649 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:49.710654 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:49.710870 (+ 216us) client_negotiation.cc:769] Sending connection context
0620 20:12:49.711032 (+ 162us) client_negotiation.cc:240] Negotiation successful
0620 20:12:49.711269 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":117,"threads_started":1}
I20260620 20:12:49.713102 25063 heartbeater.cc:347] Connected to a master server at 127.5.91.190:46067
I20260620 20:12:49.713356 25063 heartbeater.cc:464] Registering TS with master...
I20260620 20:12:49.713960 25063 heartbeater.cc:511] Master 127.5.91.190:46067 requested a full tablet report, sending...
I20260620 20:12:49.715119 24738 ts_manager.cc:194] Registered new tserver with Master: 9011b1b2d4d64262b7b7876c2ed6bc2f (127.5.91.130:33129)
I20260620 20:12:49.715668 24738 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.130@KRBTEST.COM'} at 127.5.91.130:58167
WARNING: no policy specified for kudu/127.5.91.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
Entry for principal kudu/127.5.91.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab.
I20260620 20:12:49.737160 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46067
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:37167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:12:49.848989 25068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:49.849270 25068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:49.849313 25068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:49.854019 25068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:49.854130 25068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:49.854334 25068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:12:49.861250 25068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:37167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.131.keytab
--principal=kudu/127.5.91.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:46067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:49.862468 25068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:49.863432 25068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:49.872710 25074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:49.872715 25076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:49.872973 25068 server_base.cc:1034] running on GCE node
W20260620 20:12:49.873117 25073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:49.873482 25068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:49.874097 25068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:49.875301 25068 hybrid_clock.cc:648] HybridClock initialized: now 1781986369875277 us; error 36 us; skew 500 ppm
Jun 20 20:12:49 dist-test-slave-l8h4 krb5kdc[24693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986369, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:49.878044 25068 init.cc:377] Logged in from keytab as kudu/127.5.91.131@KRBTEST.COM (short username kudu)
I20260620 20:12:49.878917 25068 webserver.cc:460] Webserver started at http://127.5.91.131:34005/ using document root <none> and password file <none>
I20260620 20:12:49.879413 25068 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:49.879467 25068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:49.879617 25068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:49.881505 25068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "5aca883b34eb41bf935cc148ee4e1aac"
format_stamp: "Formatted at 2026-06-20 20:12:49 on dist-test-slave-l8h4"
server_key: "51d98bc997de11e7d222fcee61d1ae44"
server_key_iv: "f990f98a5bd910861762132c9f0fa2a5"
server_key_version: "encryptionkey@0"
I20260620 20:12:49.882001 25068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "5aca883b34eb41bf935cc148ee4e1aac"
format_stamp: "Formatted at 2026-06-20 20:12:49 on dist-test-slave-l8h4"
server_key: "51d98bc997de11e7d222fcee61d1ae44"
server_key_iv: "f990f98a5bd910861762132c9f0fa2a5"
server_key_version: "encryptionkey@0"
I20260620 20:12:49.885450 25068 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 20:12:49.887677 25083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:49.888732 25068 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 20:12:49.888824 25068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "5aca883b34eb41bf935cc148ee4e1aac"
format_stamp: "Formatted at 2026-06-20 20:12:49 on dist-test-slave-l8h4"
server_key: "51d98bc997de11e7d222fcee61d1ae44"
server_key_iv: "f990f98a5bd910861762132c9f0fa2a5"
server_key_version: "encryptionkey@0"
I20260620 20:12:49.888918 25068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:49.906708 25068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:49.909436 25068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:49.909597 25068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:49.910174 25068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:49.910943 25068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:12:49.910986 25068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:49.911026 25068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:12:49.911054 25068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:49.919332 25068 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:45803
I20260620 20:12:49.919358 25196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:45803 every 8 connection(s)
I20260620 20:12:49.920195 25068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:12:49.923482 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 25068
I20260620 20:12:49.923607 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:12:49.923882 5486 external_mini_cluster.cc:1428] Setting key 7bf3a1e3bdf43bcdf808d6c44bfb846e
Jun 20 20:12:49 dist-test-slave-l8h4 krb5kdc[24693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986369, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.131@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:12:49.934099 24777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:49.922621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:52877 (local address 127.5.91.190:46067)
0620 20:12:49.922747 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:49.922750 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:49.923818 (+ 1068us) server_negotiation.cc:407] Connection header received
0620 20:12:49.924840 (+ 1022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:49.924843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:49.924898 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:49.924969 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:49.926077 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.926634 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.927723 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.927934 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.928017 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:49.930680 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:49.930683 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:49.930685 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:49.930708 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:49.932173 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:49.932663 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:49.932667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:49.932668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:49.932742 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:49.933039 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:49.933043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:49.933044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:49.933228 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:49.933311 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:49.933794 (+ 483us) server_negotiation.cc:299] Negotiation successful
0620 20:12:49.933914 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 20:12:49.934952 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:49.922937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46067 (local address 127.5.91.131:52877)
0620 20:12:49.923675 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:49.923727 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:49.924641 (+ 914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:49.925148 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:49.925154 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:49.925492 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:49.925949 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:49.925961 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.926753 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.926756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:49.927613 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:49.927622 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.928064 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.928068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:49.928230 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:49.928954 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:12:49.928972 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:49.930517 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:49.932294 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:49.932300 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:49.932312 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:49.932572 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:49.932824 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:12:49.932826 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:12:49.932828 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:12:49.932956 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:12:49.933325 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:49.933331 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:49.933574 (+ 243us) client_negotiation.cc:769] Sending connection context
0620 20:12:49.933738 (+ 164us) client_negotiation.cc:240] Negotiation successful
0620 20:12:49.934062 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":122,"threads_started":1}
I20260620 20:12:49.936091 25197 heartbeater.cc:347] Connected to a master server at 127.5.91.190:46067
I20260620 20:12:49.936317 25197 heartbeater.cc:464] Registering TS with master...
I20260620 20:12:49.936789 25197 heartbeater.cc:511] Master 127.5.91.190:46067 requested a full tablet report, sending...
I20260620 20:12:49.937770 24738 ts_manager.cc:194] Registered new tserver with Master: 5aca883b34eb41bf935cc148ee4e1aac (127.5.91.131:45803)
I20260620 20:12:49.938335 24738 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.91.131@KRBTEST.COM'} at 127.5.91.131:52877
I20260620 20:12:49.949759 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 20:12:49.959137 24777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:49.951704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49914 (local address 127.5.91.190:46067)
0620 20:12:49.951831 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:49.951835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:49.951955 (+ 120us) server_negotiation.cc:407] Connection header received
0620 20:12:49.952056 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:49.952058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:49.952105 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:49.952166 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:49.952923 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.953535 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.954508 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.954736 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.954814 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:49.956189 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:49.956193 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:49.956196 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:49.956223 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:49.957694 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:49.958084 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:49.958087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:49.958088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:49.958209 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:49.958411 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:49.958412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:49.958413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:49.958631 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:49.958731 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:49.958905 (+ 174us) server_negotiation.cc:299] Negotiation successful
0620 20:12:49.958994 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 20:12:49.962386 24738 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49914:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:12:49.965342 24738 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:12:49.980482 25211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:49.974266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44438 (local address 127.5.91.131:45803)
0620 20:12:49.974755 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:49.974770 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:49.975025 (+ 255us) server_negotiation.cc:407] Connection header received
0620 20:12:49.975320 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:49.975326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:49.975537 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:49.976054 (+ 517us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:49.976192 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.977413 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.979160 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.979812 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.980018 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:49.980174 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:49.980223 (+ 49us) server_negotiation.cc:299] Negotiation successful
0620 20:12:49.980319 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":124,"threads_started":1}
I20260620 20:12:49.980518 25208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:49.973722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:45803 (local address 127.0.0.1:44438)
0620 20:12:49.974766 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:49.974893 (+ 127us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:49.975102 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:49.975713 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:49.975719 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:49.975742 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:49.975886 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:49.975893 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.977567 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.977569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:49.979011 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:49.979021 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.979931 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.979934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:49.980047 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:49.980086 (+ 39us) client_negotiation.cc:769] Sending connection context
0620 20:12:49.980188 (+ 102us) client_negotiation.cc:240] Negotiation successful
0620 20:12:49.980371 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"mutex_wait_us":55,"thread_start_us":91,"threads_started":1}
I20260620 20:12:49.981097 25210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:49.974038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57958 (local address 127.5.91.129:37349)
0620 20:12:49.975608 (+ 1570us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:49.975621 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:49.975641 (+ 20us) server_negotiation.cc:407] Connection header received
0620 20:12:49.975710 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:49.975715 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:49.975898 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:49.976053 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:49.976265 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.977412 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.979099 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.979787 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.979898 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:49.980112 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:49.980845 (+ 733us) server_negotiation.cc:299] Negotiation successful
0620 20:12:49.980957 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1470,"thread_start_us":145,"threads_started":1}
I20260620 20:12:49.981302 25212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:49.973852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:37349 (local address 127.0.0.1:57958)
0620 20:12:49.975261 (+ 1409us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:49.975286 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:49.975380 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:49.976028 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:49.976031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:49.976049 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:49.976149 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:49.976156 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.977777 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.977780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:49.978990 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:49.978999 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.979917 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.979921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:49.980329 (+ 408us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:49.980349 (+ 20us) client_negotiation.cc:769] Sending connection context
0620 20:12:49.980409 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 20:12:49.980469 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1332,"mutex_wait_us":27,"thread_start_us":86,"threads_started":1}
I20260620 20:12:49.982620 25209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:49.974515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:33129 (local address 127.0.0.1:43484)
0620 20:12:49.975704 (+ 1189us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:49.975719 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:49.975829 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:49.977572 (+ 1743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:49.977575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:49.977595 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:49.977670 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:49.977674 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.979445 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.979448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:49.980952 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:49.980962 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.982229 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.982233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:49.982327 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:49.982350 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 20:12:49.982430 (+ 80us) client_negotiation.cc:240] Negotiation successful
0620 20:12:49.982497 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1126,"mutex_wait_us":1,"thread_start_us":86,"threads_started":1}
I20260620 20:12:49.983057 25213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:49.974909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43484 (local address 127.5.91.130:33129)
0620 20:12:49.975536 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:49.975550 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:49.975816 (+ 266us) server_negotiation.cc:407] Connection header received
0620 20:12:49.975954 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:49.975960 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:49.976132 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:49.976281 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:49.977777 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.979323 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.981094 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:49.981912 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:49.982012 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:49.982167 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:49.982831 (+ 664us) server_negotiation.cc:299] Negotiation successful
0620 20:12:49.982923 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":94,"threads_started":1}
I20260620 20:12:49.983520 25131 tablet_service.cc:1467] Processing CreateTablet for tablet 24fcc501596a4430b9badf5599aa6f5d (DEFAULT_TABLE table=test-table [id=997c49f74906472cae86a23e556c0d3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:12:49.984174 24863 tablet_service.cc:1467] Processing CreateTablet for tablet 24fcc501596a4430b9badf5599aa6f5d (DEFAULT_TABLE table=test-table [id=997c49f74906472cae86a23e556c0d3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:12:49.984642 25131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24fcc501596a4430b9badf5599aa6f5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:49.984938 24863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24fcc501596a4430b9badf5599aa6f5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:49.985477 24997 tablet_service.cc:1467] Processing CreateTablet for tablet 24fcc501596a4430b9badf5599aa6f5d (DEFAULT_TABLE table=test-table [id=997c49f74906472cae86a23e556c0d3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:12:49.986419 24997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24fcc501596a4430b9badf5599aa6f5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:49.991569 25214 tablet_bootstrap.cc:492] T 24fcc501596a4430b9badf5599aa6f5d P 0b378aedf5c14370a10cf6757b09f8d5: Bootstrap starting.
I20260620 20:12:49.993608 25214 tablet_bootstrap.cc:654] T 24fcc501596a4430b9badf5599aa6f5d P 0b378aedf5c14370a10cf6757b09f8d5: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:49.993901 25216 tablet_bootstrap.cc:492] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f: Bootstrap starting.
I20260620 20:12:49.994436 25215 tablet_bootstrap.cc:492] T 24fcc501596a4430b9badf5599aa6f5d P 5aca883b34eb41bf935cc148ee4e1aac: Bootstrap starting.
I20260620 20:12:49.994752 25214 log.cc:824] T 24fcc501596a4430b9badf5599aa6f5d P 0b378aedf5c14370a10cf6757b09f8d5: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:49.996285 25216 tablet_bootstrap.cc:654] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:49.996356 25215 tablet_bootstrap.cc:654] T 24fcc501596a4430b9badf5599aa6f5d P 5aca883b34eb41bf935cc148ee4e1aac: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:49.996608 25214 tablet_bootstrap.cc:492] T 24fcc501596a4430b9badf5599aa6f5d P 0b378aedf5c14370a10cf6757b09f8d5: No bootstrap required, opened a new log
I20260620 20:12:49.996793 25214 ts_tablet_manager.cc:1397] T 24fcc501596a4430b9badf5599aa6f5d P 0b378aedf5c14370a10cf6757b09f8d5: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 20:12:49.997120 25216 log.cc:824] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:49.997416 25215 log.cc:824] T 24fcc501596a4430b9badf5599aa6f5d P 5aca883b34eb41bf935cc148ee4e1aac: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:49.998811 25216 tablet_bootstrap.cc:492] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f: No bootstrap required, opened a new log
I20260620 20:12:49.998976 25216 ts_tablet_manager.cc:1397] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 20:12:49.999186 25215 tablet_bootstrap.cc:492] T 24fcc501596a4430b9badf5599aa6f5d P 5aca883b34eb41bf935cc148ee4e1aac: No bootstrap required, opened a new log
I20260620 20:12:49.999361 25215 ts_tablet_manager.cc:1397] T 24fcc501596a4430b9badf5599aa6f5d P 5aca883b34eb41bf935cc148ee4e1aac: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 20:12:50.000427 25214 raft_consensus.cc:348] T 24fcc501596a4430b9badf5599aa6f5d P 0b378aedf5c14370a10cf6757b09f8d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aca883b34eb41bf935cc148ee4e1aac" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 45803 } } peers { permanent_uuid: "0b378aedf5c14370a10cf6757b09f8d5" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 37349 } } peers { permanent_uuid: "9011b1b2d4d64262b7b7876c2ed6bc2f" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 33129 } }
I20260620 20:12:50.000715 25214 raft_consensus.cc:374] T 24fcc501596a4430b9badf5599aa6f5d P 0b378aedf5c14370a10cf6757b09f8d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:50.000933 25214 raft_consensus.cc:729] T 24fcc501596a4430b9badf5599aa6f5d P 0b378aedf5c14370a10cf6757b09f8d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b378aedf5c14370a10cf6757b09f8d5, State: Initialized, Role: FOLLOWER
I20260620 20:12:50.001520 25214 consensus_queue.cc:260] T 24fcc501596a4430b9badf5599aa6f5d P 0b378aedf5c14370a10cf6757b09f8d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aca883b34eb41bf935cc148ee4e1aac" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 45803 } } peers { permanent_uuid: "0b378aedf5c14370a10cf6757b09f8d5" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 37349 } } peers { permanent_uuid: "9011b1b2d4d64262b7b7876c2ed6bc2f" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 33129 } }
I20260620 20:12:50.002161 25216 raft_consensus.cc:348] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aca883b34eb41bf935cc148ee4e1aac" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 45803 } } peers { permanent_uuid: "0b378aedf5c14370a10cf6757b09f8d5" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 37349 } } peers { permanent_uuid: "9011b1b2d4d64262b7b7876c2ed6bc2f" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 33129 } }
I20260620 20:12:50.002725 24929 heartbeater.cc:503] Master 127.5.91.190:46067 was elected leader, sending a full tablet report...
I20260620 20:12:50.002966 25216 raft_consensus.cc:374] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:50.003067 25216 raft_consensus.cc:729] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9011b1b2d4d64262b7b7876c2ed6bc2f, State: Initialized, Role: FOLLOWER
I20260620 20:12:50.002734 25214 ts_tablet_manager.cc:1428] T 24fcc501596a4430b9badf5599aa6f5d P 0b378aedf5c14370a10cf6757b09f8d5: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260620 20:12:50.003609 25216 consensus_queue.cc:260] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aca883b34eb41bf935cc148ee4e1aac" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 45803 } } peers { permanent_uuid: "0b378aedf5c14370a10cf6757b09f8d5" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 37349 } } peers { permanent_uuid: "9011b1b2d4d64262b7b7876c2ed6bc2f" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 33129 } }
I20260620 20:12:50.003983 25215 raft_consensus.cc:348] T 24fcc501596a4430b9badf5599aa6f5d P 5aca883b34eb41bf935cc148ee4e1aac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aca883b34eb41bf935cc148ee4e1aac" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 45803 } } peers { permanent_uuid: "0b378aedf5c14370a10cf6757b09f8d5" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 37349 } } peers { permanent_uuid: "9011b1b2d4d64262b7b7876c2ed6bc2f" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 33129 } }
I20260620 20:12:50.004278 25215 raft_consensus.cc:374] T 24fcc501596a4430b9badf5599aa6f5d P 5aca883b34eb41bf935cc148ee4e1aac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:50.004339 25215 raft_consensus.cc:729] T 24fcc501596a4430b9badf5599aa6f5d P 5aca883b34eb41bf935cc148ee4e1aac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aca883b34eb41bf935cc148ee4e1aac, State: Initialized, Role: FOLLOWER
I20260620 20:12:50.004545 25063 heartbeater.cc:503] Master 127.5.91.190:46067 was elected leader, sending a full tablet report...
I20260620 20:12:50.004590 25216 ts_tablet_manager.cc:1428] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260620 20:12:50.004738 25215 consensus_queue.cc:260] T 24fcc501596a4430b9badf5599aa6f5d P 5aca883b34eb41bf935cc148ee4e1aac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aca883b34eb41bf935cc148ee4e1aac" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 45803 } } peers { permanent_uuid: "0b378aedf5c14370a10cf6757b09f8d5" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 37349 } } peers { permanent_uuid: "9011b1b2d4d64262b7b7876c2ed6bc2f" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 33129 } }
I20260620 20:12:50.005373 25197 heartbeater.cc:503] Master 127.5.91.190:46067 was elected leader, sending a full tablet report...
I20260620 20:12:50.005678 25215 ts_tablet_manager.cc:1428] T 24fcc501596a4430b9badf5599aa6f5d P 5aca883b34eb41bf935cc148ee4e1aac: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260620 20:12:50.165268 25221 raft_consensus.cc:482] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:12:50.165453 25221 raft_consensus.cc:504] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aca883b34eb41bf935cc148ee4e1aac" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 45803 } } peers { permanent_uuid: "0b378aedf5c14370a10cf6757b09f8d5" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 37349 } } peers { permanent_uuid: "9011b1b2d4d64262b7b7876c2ed6bc2f" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 33129 } }
I20260620 20:12:50.166587 25221 leader_election.cc:288] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5aca883b34eb41bf935cc148ee4e1aac (127.5.91.131:45803), 0b378aedf5c14370a10cf6757b09f8d5 (127.5.91.129:37349)
W20260620 20:12:50.171218 25198 tablet.cc:2369] T 24fcc501596a4430b9badf5599aa6f5d P 5aca883b34eb41bf935cc148ee4e1aac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:12:50.171447 25065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:50.166784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:45803 (local address 127.5.91.130:37927)
0620 20:12:50.167077 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:50.167091 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:50.167182 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:50.167444 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:50.167446 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:50.167459 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:50.167581 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:50.167588 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:50.168594 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:50.168598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:50.170036 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:50.170043 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:50.170933 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:50.170937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:50.171163 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:50.171180 (+ 17us) client_negotiation.cc:769] Sending connection context
0620 20:12:50.171243 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 20:12:50.171312 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":174}
I20260620 20:12:50.171466 25210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:50.167028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:50517 (local address 127.5.91.129:37349)
0620 20:12:50.167135 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:50.167139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:50.167315 (+ 176us) server_negotiation.cc:407] Connection header received
0620 20:12:50.167402 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:50.167405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:50.167450 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:50.167516 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:50.167723 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:50.168511 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:50.170182 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:50.170801 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:50.170922 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:50.171012 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:50.171296 (+ 284us) server_negotiation.cc:299] Negotiation successful
0620 20:12:50.171370 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 20:12:50.171458 25211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:50.166879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:37927 (local address 127.5.91.131:45803)
0620 20:12:50.166990 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:50.166994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:50.167248 (+ 254us) server_negotiation.cc:407] Connection header received
0620 20:12:50.167297 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:50.167300 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:50.167348 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:50.167413 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:50.167695 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:50.168462 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:50.170196 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:50.170832 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:50.170926 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:50.170972 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:50.171281 (+ 309us) server_negotiation.cc:299] Negotiation successful
0620 20:12:50.171355 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 20:12:50.171993 25223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:50.166863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:37349 (local address 127.5.91.130:50517)
0620 20:12:50.167221 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:50.167232 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:50.167327 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:50.167527 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:50.167529 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:50.167542 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:50.167645 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:50.167651 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:50.168613 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:50.168615 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:50.170031 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:50.170042 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:50.170933 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:50.170937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:50.171098 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:50.171172 (+ 74us) client_negotiation.cc:769] Sending connection context
0620 20:12:50.171787 (+ 615us) client_negotiation.cc:240] Negotiation successful
0620 20:12:50.171856 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":144,"thread_start_us":117,"threads_started":1}
I20260620 20:12:50.172170 25151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24fcc501596a4430b9badf5599aa6f5d" candidate_uuid: "9011b1b2d4d64262b7b7876c2ed6bc2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aca883b34eb41bf935cc148ee4e1aac" is_pre_election: true
I20260620 20:12:50.172470 25151 raft_consensus.cc:2454] T 24fcc501596a4430b9badf5599aa6f5d P 5aca883b34eb41bf935cc148ee4e1aac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9011b1b2d4d64262b7b7876c2ed6bc2f in term 0.
I20260620 20:12:50.172525 24883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24fcc501596a4430b9badf5599aa6f5d" candidate_uuid: "9011b1b2d4d64262b7b7876c2ed6bc2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b378aedf5c14370a10cf6757b09f8d5" is_pre_election: true
I20260620 20:12:50.172765 24883 raft_consensus.cc:2454] T 24fcc501596a4430b9badf5599aa6f5d P 0b378aedf5c14370a10cf6757b09f8d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9011b1b2d4d64262b7b7876c2ed6bc2f in term 0.
I20260620 20:12:50.172917 24953 leader_election.cc:302] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aca883b34eb41bf935cc148ee4e1aac, 9011b1b2d4d64262b7b7876c2ed6bc2f; no voters:
I20260620 20:12:50.173197 25221 raft_consensus.cc:2781] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:12:50.173267 25221 raft_consensus.cc:482] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:12:50.173314 25221 raft_consensus.cc:3037] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:50.174361 25221 raft_consensus.cc:504] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aca883b34eb41bf935cc148ee4e1aac" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 45803 } } peers { permanent_uuid: "0b378aedf5c14370a10cf6757b09f8d5" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 37349 } } peers { permanent_uuid: "9011b1b2d4d64262b7b7876c2ed6bc2f" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 33129 } }
I20260620 20:12:50.174688 25221 leader_election.cc:288] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f [CANDIDATE]: Term 1 election: Requested vote from peers 5aca883b34eb41bf935cc148ee4e1aac (127.5.91.131:45803), 0b378aedf5c14370a10cf6757b09f8d5 (127.5.91.129:37349)
I20260620 20:12:50.175107 25151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24fcc501596a4430b9badf5599aa6f5d" candidate_uuid: "9011b1b2d4d64262b7b7876c2ed6bc2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aca883b34eb41bf935cc148ee4e1aac"
I20260620 20:12:50.175105 24883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24fcc501596a4430b9badf5599aa6f5d" candidate_uuid: "9011b1b2d4d64262b7b7876c2ed6bc2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b378aedf5c14370a10cf6757b09f8d5"
I20260620 20:12:50.175222 25151 raft_consensus.cc:3037] T 24fcc501596a4430b9badf5599aa6f5d P 5aca883b34eb41bf935cc148ee4e1aac [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:50.175222 24883 raft_consensus.cc:3037] T 24fcc501596a4430b9badf5599aa6f5d P 0b378aedf5c14370a10cf6757b09f8d5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:50.176375 25151 raft_consensus.cc:2454] T 24fcc501596a4430b9badf5599aa6f5d P 5aca883b34eb41bf935cc148ee4e1aac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9011b1b2d4d64262b7b7876c2ed6bc2f in term 1.
I20260620 20:12:50.176375 24883 raft_consensus.cc:2454] T 24fcc501596a4430b9badf5599aa6f5d P 0b378aedf5c14370a10cf6757b09f8d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9011b1b2d4d64262b7b7876c2ed6bc2f in term 1.
I20260620 20:12:50.176693 24952 leader_election.cc:302] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0b378aedf5c14370a10cf6757b09f8d5, 5aca883b34eb41bf935cc148ee4e1aac, 9011b1b2d4d64262b7b7876c2ed6bc2f; no voters:
I20260620 20:12:50.176892 25221 raft_consensus.cc:2781] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:12:50.177066 25221 raft_consensus.cc:686] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f [term 1 LEADER]: Becoming Leader. State: Replica: 9011b1b2d4d64262b7b7876c2ed6bc2f, State: Running, Role: LEADER
I20260620 20:12:50.177397 25221 consensus_queue.cc:237] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aca883b34eb41bf935cc148ee4e1aac" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 45803 } } peers { permanent_uuid: "0b378aedf5c14370a10cf6757b09f8d5" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 37349 } } peers { permanent_uuid: "9011b1b2d4d64262b7b7876c2ed6bc2f" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 33129 } }
I20260620 20:12:50.181129 24733 catalog_manager.cc:5495] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f reported cstate change: term changed from 0 to 1, leader changed from <none> to 9011b1b2d4d64262b7b7876c2ed6bc2f (127.5.91.130). New cstate: current_term: 1 leader_uuid: "9011b1b2d4d64262b7b7876c2ed6bc2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aca883b34eb41bf935cc148ee4e1aac" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 45803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b378aedf5c14370a10cf6757b09f8d5" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 37349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9011b1b2d4d64262b7b7876c2ed6bc2f" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 33129 } health_report { overall_health: HEALTHY } } }
W20260620 20:12:50.199256 25064 tablet.cc:2369] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:12:50.214721 25213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:50.211427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43490 (local address 127.5.91.130:33129)
0620 20:12:50.211547 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:50.211551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:50.211565 (+ 14us) server_negotiation.cc:407] Connection header received
0620 20:12:50.211640 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:50.211642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:50.211700 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:50.211796 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:12:50.212091 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:50.212593 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:50.213400 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:50.213613 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:50.213680 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:50.213891 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:12:50.214348 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:12:50.214434 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:50.214513 (+ 79us) server_negotiation.cc:299] Negotiation successful
0620 20:12:50.214561 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 20:12:50.220047 25151 raft_consensus.cc:1261] T 24fcc501596a4430b9badf5599aa6f5d P 5aca883b34eb41bf935cc148ee4e1aac [term 1 FOLLOWER]: Refusing update from remote peer 9011b1b2d4d64262b7b7876c2ed6bc2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:12:50.220047 24883 raft_consensus.cc:1261] T 24fcc501596a4430b9badf5599aa6f5d P 0b378aedf5c14370a10cf6757b09f8d5 [term 1 FOLLOWER]: Refusing update from remote peer 9011b1b2d4d64262b7b7876c2ed6bc2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:12:50.220705 25224 consensus_queue.cc:1035] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b378aedf5c14370a10cf6757b09f8d5" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:12:50.220916 25221 consensus_queue.cc:1035] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aca883b34eb41bf935cc148ee4e1aac" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 45803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:12:50.231745 25228 mvcc.cc:205] Tried to move back new op lower bound from 7299016172414607360 to 7299016172249186304. Current Snapshot: MvccSnapshot[applied={T|T < 7299016172414607360}]
I20260620 20:12:50.238929 24733 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:12:50.239117 24733 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:12:50.241577 24733 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e8638b92dce9409fa1a2c8ab9a8daa5c: Sending DeleteTablet for 3 replicas of tablet 24fcc501596a4430b9badf5599aa6f5d
I20260620 20:12:50.242504 24863 tablet_service.cc:1514] Processing DeleteTablet for tablet 24fcc501596a4430b9badf5599aa6f5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:12:50 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:57958
I20260620 20:12:50.242516 25131 tablet_service.cc:1514] Processing DeleteTablet for tablet 24fcc501596a4430b9badf5599aa6f5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:12:50 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:44438
I20260620 20:12:50.243227 24997 tablet_service.cc:1514] Processing DeleteTablet for tablet 24fcc501596a4430b9badf5599aa6f5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:12:50 UTC) from {username='kudu', principal='kudu/127.5.91.190@KRBTEST.COM'} at 127.0.0.1:43484
I20260620 20:12:50.243659 25237 tablet_replica.cc:330] T 24fcc501596a4430b9badf5599aa6f5d P 0b378aedf5c14370a10cf6757b09f8d5: stopping tablet replica
I20260620 20:12:50.243853 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 24800
I20260620 20:12:50.245218 25239 tablet_replica.cc:330] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f: stopping tablet replica
I20260620 20:12:50.245472 25239 raft_consensus.cc:2229] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f [term 1 LEADER]: Raft consensus shutting down.
I20260620 20:12:50.245790 25239 raft_consensus.cc:2258] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:12:50.247090 25239 ts_tablet_manager.cc:1905] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:12:50.249619 25239 ts_tablet_manager.cc:1918] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:12:50.249722 25239 log.cc:1196] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-1/wal/wals/24fcc501596a4430b9badf5599aa6f5d
I20260620 20:12:50.250000 25239 ts_tablet_manager.cc:1939] T 24fcc501596a4430b9badf5599aa6f5d P 9011b1b2d4d64262b7b7876c2ed6bc2f: Deleting consensus metadata
I20260620 20:12:50.250114 25238 tablet_replica.cc:330] T 24fcc501596a4430b9badf5599aa6f5d P 5aca883b34eb41bf935cc148ee4e1aac: stopping tablet replica
I20260620 20:12:50.250466 25238 raft_consensus.cc:2229] T 24fcc501596a4430b9badf5599aa6f5d P 5aca883b34eb41bf935cc148ee4e1aac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:12:50.250818 25238 raft_consensus.cc:2258] T 24fcc501596a4430b9badf5599aa6f5d P 5aca883b34eb41bf935cc148ee4e1aac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:12:50.251593 24723 catalog_manager.cc:4841] TS 9011b1b2d4d64262b7b7876c2ed6bc2f (127.5.91.130:33129): tablet 24fcc501596a4430b9badf5599aa6f5d (table test-table [id=997c49f74906472cae86a23e556c0d3f]) successfully deleted
I20260620 20:12:50.252399 25238 ts_tablet_manager.cc:1905] T 24fcc501596a4430b9badf5599aa6f5d P 5aca883b34eb41bf935cc148ee4e1aac: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 20:12:50.253005 24724 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.91.129:37349 (error 108)
I20260620 20:12:50.253782 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 24934
I20260620 20:12:50.254010 25209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:50.253727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:37349 (local address 127.0.0.1:57962)
0620 20:12:50.253840 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:50.253903 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.129:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 20:12:50.254675 24724 catalog_manager.cc:4584] TS 0b378aedf5c14370a10cf6757b09f8d5 (127.5.91.129:37349): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 24fcc501596a4430b9badf5599aa6f5d: Network error: Client connection negotiation failed: client connection to 127.5.91.129:37349: connect: Connection refused (error 111)
I20260620 20:12:50.257115 25238 ts_tablet_manager.cc:1918] T 24fcc501596a4430b9badf5599aa6f5d P 5aca883b34eb41bf935cc148ee4e1aac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:12:50.257212 25238 log.cc:1196] T 24fcc501596a4430b9badf5599aa6f5d P 5aca883b34eb41bf935cc148ee4e1aac: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/24fcc501596a4430b9badf5599aa6f5d
I20260620 20:12:50.257483 25238 ts_tablet_manager.cc:1939] T 24fcc501596a4430b9badf5599aa6f5d P 5aca883b34eb41bf935cc148ee4e1aac: Deleting consensus metadata
I20260620 20:12:50.258878 24725 catalog_manager.cc:4841] TS 5aca883b34eb41bf935cc148ee4e1aac (127.5.91.131:45803): tablet 24fcc501596a4430b9badf5599aa6f5d (table test-table [id=997c49f74906472cae86a23e556c0d3f]) successfully deleted
I20260620 20:12:50.259845 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 25068
I20260620 20:12:50.265125 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 24706
2026-06-20T20:12:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2300 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-20T20:12:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:12:50Z Disabled control of system clock
I20260620 20:12:50.294981 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:37373
--webserver_interface=127.5.91.190
--webserver_port=0
--builtin_ntp_servers=127.5.91.148:35323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:37373
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 20:12:50.401911 25244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:50.402207 25244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:50.402247 25244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:50.406286 25244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:12:50.406350 25244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:50.406368 25244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:50.406386 25244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:12:50.406406 25244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:12:50.411249 25244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:35323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:37373
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:37373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:50.412302 25244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:50.413654 25244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 20:12:50.421959 25244 server_base.cc:1034] running on GCE node
W20260620 20:12:50.421976 25249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:50.422031 25250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:50.422127 25252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:50.422834 25244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:50.423904 25244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:50.425077 25244 hybrid_clock.cc:648] HybridClock initialized: now 1781986370425054 us; error 41 us; skew 500 ppm
I20260620 20:12:50.427083 25244 webserver.cc:460] Webserver started at http://127.5.91.190:37273/ using document root <none> and password file <none>
I20260620 20:12:50.427634 25244 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:50.427699 25244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:50.427892 25244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:50.429873 25244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "418b2e3cd8744198b67394614029b898"
format_stamp: "Formatted at 2026-06-20 20:12:50 on dist-test-slave-l8h4"
server_key: "c90c37544bfb2c8f603e4f0e0da3ce2c"
server_key_iv: "0d7953aadec2d0a951fab6ec713e524b"
server_key_version: "encryptionkey@0"
I20260620 20:12:50.430429 25244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "418b2e3cd8744198b67394614029b898"
format_stamp: "Formatted at 2026-06-20 20:12:50 on dist-test-slave-l8h4"
server_key: "c90c37544bfb2c8f603e4f0e0da3ce2c"
server_key_iv: "0d7953aadec2d0a951fab6ec713e524b"
server_key_version: "encryptionkey@0"
I20260620 20:12:50.433914 25244 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260620 20:12:50.436347 25258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:50.437512 25244 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260620 20:12:50.437630 25244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "418b2e3cd8744198b67394614029b898"
format_stamp: "Formatted at 2026-06-20 20:12:50 on dist-test-slave-l8h4"
server_key: "c90c37544bfb2c8f603e4f0e0da3ce2c"
server_key_iv: "0d7953aadec2d0a951fab6ec713e524b"
server_key_version: "encryptionkey@0"
I20260620 20:12:50.437721 25244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:50.453872 25244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:50.454604 25244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:50.454777 25244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:50.461424 25244 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:37373
I20260620 20:12:50.461440 25310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:37373 every 8 connection(s)
I20260620 20:12:50.462381 25244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:12:50.465389 25311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:50.470719 25311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898: Bootstrap starting.
I20260620 20:12:50.470971 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 25244
I20260620 20:12:50.471087 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:12:50.471341 5486 external_mini_cluster.cc:1428] Setting key e3261d7e61d106a54a1465242789e406
I20260620 20:12:50.473146 25311 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:50.473826 25311 log.cc:824] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:50.475826 25311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898: No bootstrap required, opened a new log
I20260620 20:12:50.479285 25311 raft_consensus.cc:348] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "418b2e3cd8744198b67394614029b898" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 37373 } }
I20260620 20:12:50.479547 25311 raft_consensus.cc:374] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:50.479601 25311 raft_consensus.cc:729] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 418b2e3cd8744198b67394614029b898, State: Initialized, Role: FOLLOWER
I20260620 20:12:50.480027 25311 consensus_queue.cc:260] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "418b2e3cd8744198b67394614029b898" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 37373 } }
I20260620 20:12:50.480160 25311 raft_consensus.cc:388] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:12:50.480216 25311 raft_consensus.cc:482] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:12:50.480306 25311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:50.480455 25314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:50.472638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43600 (local address 127.5.91.190:37373)
0620 20:12:50.473324 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:50.473339 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:50.473376 (+ 37us) server_negotiation.cc:407] Connection header received
0620 20:12:50.474058 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:50.474075 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:50.474544 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:50.474903 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:50.475316 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:50.476289 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:50.477132 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:50.477609 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:50.477752 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:50.478614 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:50.478619 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:50.478632 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:12:50.478995 (+ 363us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:50.479049 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:12:50.479068 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:50.479156 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:50.479289 (+ 133us) server_negotiation.cc:299] Negotiation successful
0620 20:12:50.479650 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":121,"threads_started":1}
I20260620 20:12:50.481249 25311 raft_consensus.cc:504] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "418b2e3cd8744198b67394614029b898" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 37373 } }
I20260620 20:12:50.481555 25311 leader_election.cc:302] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 418b2e3cd8744198b67394614029b898; no voters:
I20260620 20:12:50.481825 25311 leader_election.cc:288] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:12:50.481945 25316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:12:50.482187 25316 raft_consensus.cc:686] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898 [term 1 LEADER]: Becoming Leader. State: Replica: 418b2e3cd8744198b67394614029b898, State: Running, Role: LEADER
I20260620 20:12:50.482488 25316 consensus_queue.cc:237] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "418b2e3cd8744198b67394614029b898" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 37373 } }
I20260620 20:12:50.482977 25311 sys_catalog.cc:564] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:12:50.483927 25317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "418b2e3cd8744198b67394614029b898" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "418b2e3cd8744198b67394614029b898" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 37373 } } }
I20260620 20:12:50.484048 25317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898 [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:50.484407 25325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:12:50.484555 25318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 418b2e3cd8744198b67394614029b898. Latest consensus state: current_term: 1 leader_uuid: "418b2e3cd8744198b67394614029b898" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "418b2e3cd8744198b67394614029b898" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 37373 } } }
I20260620 20:12:50.484663 25318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898 [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:50.487080 25325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:12:50.491837 25325 catalog_manager.cc:1347] Generated new cluster ID: 6099dfd81acf427f96571251227ab48b
I20260620 20:12:50.491897 25325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:12:50.511669 25325 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:12:50.512614 25325 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:12:50.517443 25325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898: Generated new TSK 0
I20260620 20:12:50.518049 25325 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:12:50.524066 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.129:0
--local_ip_for_outbound_sockets=127.5.91.129
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:37373
--builtin_ntp_servers=127.5.91.148:35323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 20:12:50.635354 25335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:50.635605 25335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:50.635641 25335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:50.639818 25335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:50.639875 25335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:50.639931 25335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.129
I20260620 20:12:50.644800 25335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:35323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.91.129
--webserver_port=0
--tserver_master_addrs=127.5.91.190:37373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.129
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:50.645787 25335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:50.647102 25335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:50.656196 25341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:50.656215 25343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:50.656181 25340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:50.656394 25335 server_base.cc:1034] running on GCE node
I20260620 20:12:50.656889 25335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:50.657462 25335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:50.658618 25335 hybrid_clock.cc:648] HybridClock initialized: now 1781986370658599 us; error 29 us; skew 500 ppm
I20260620 20:12:50.660372 25335 webserver.cc:460] Webserver started at http://127.5.91.129:35379/ using document root <none> and password file <none>
I20260620 20:12:50.660900 25335 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:50.660969 25335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:50.661145 25335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:50.663118 25335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-0/data/instance:
uuid: "34eeb5a72d574c72b27b17c0befe5ff9"
format_stamp: "Formatted at 2026-06-20 20:12:50 on dist-test-slave-l8h4"
server_key: "019fe2924eb4930346d9c4aed7396fc6"
server_key_iv: "ffe318694314f7f87e26e7bf27b1beb2"
server_key_version: "encryptionkey@0"
I20260620 20:12:50.663617 25335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance:
uuid: "34eeb5a72d574c72b27b17c0befe5ff9"
format_stamp: "Formatted at 2026-06-20 20:12:50 on dist-test-slave-l8h4"
server_key: "019fe2924eb4930346d9c4aed7396fc6"
server_key_iv: "ffe318694314f7f87e26e7bf27b1beb2"
server_key_version: "encryptionkey@0"
I20260620 20:12:50.667032 25335 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260620 20:12:50.669193 25349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:50.670360 25335 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260620 20:12:50.670459 25335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-0/wal
uuid: "34eeb5a72d574c72b27b17c0befe5ff9"
format_stamp: "Formatted at 2026-06-20 20:12:50 on dist-test-slave-l8h4"
server_key: "019fe2924eb4930346d9c4aed7396fc6"
server_key_iv: "ffe318694314f7f87e26e7bf27b1beb2"
server_key_version: "encryptionkey@0"
I20260620 20:12:50.670542 25335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:50.691524 25335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:50.692198 25335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:50.692473 25335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:50.693066 25335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:50.694100 25335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:12:50.694167 25335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:50.694223 25335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:12:50.694249 25335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:50.702392 25335 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.129:40683
I20260620 20:12:50.702427 25462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.129:40683 every 8 connection(s)
I20260620 20:12:50.703262 25335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-0/data/info.pb
I20260620 20:12:50.710325 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 25335
I20260620 20:12:50.710456 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-0/wal/instance
I20260620 20:12:50.710776 5486 external_mini_cluster.cc:1428] Setting key 2bb5c8b8649eb9296cf3ee84fd1345ec
I20260620 20:12:50.712869 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.130:0
--local_ip_for_outbound_sockets=127.5.91.130
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:37373
--builtin_ntp_servers=127.5.91.148:35323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 20:12:50.713734 25314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:50.705005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.129:53891 (local address 127.5.91.190:37373)
0620 20:12:50.705169 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:50.705174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:50.706224 (+ 1050us) server_negotiation.cc:407] Connection header received
0620 20:12:50.707801 (+ 1577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:50.707803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:50.707851 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:50.707927 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:50.709403 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:50.709943 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:50.711707 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:50.712029 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:50.712132 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:50.713306 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:50.713309 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:50.713311 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:12:50.713322 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:50.713341 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:12:50.713349 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:50.713406 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:50.713570 (+ 164us) server_negotiation.cc:299] Negotiation successful
0620 20:12:50.713623 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 20:12:50.714792 25465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:50.705425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:37373 (local address 127.5.91.129:53891)
0620 20:12:50.706073 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:50.706114 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:50.707532 (+ 1418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:50.708308 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:50.708319 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:50.708447 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:50.709198 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:50.709216 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:50.710288 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:50.710292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:50.711574 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:50.711586 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:50.712206 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:50.712211 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:50.712396 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:50.713022 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 20:12:50.713024 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:50.713081 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 20:12:50.713160 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:50.713423 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:50.713429 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:50.713434 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 20:12:50.713567 (+ 133us) client_negotiation.cc:240] Negotiation successful
0620 20:12:50.713863 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":112,"threads_started":1}
I20260620 20:12:50.715889 25463 heartbeater.cc:347] Connected to a master server at 127.5.91.190:37373
I20260620 20:12:50.716118 25463 heartbeater.cc:464] Registering TS with master...
I20260620 20:12:50.716710 25463 heartbeater.cc:511] Master 127.5.91.190:37373 requested a full tablet report, sending...
I20260620 20:12:50.718219 25275 ts_manager.cc:194] Registered new tserver with Master: 34eeb5a72d574c72b27b17c0befe5ff9 (127.5.91.129:40683)
I20260620 20:12:50.719398 25275 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.91.129:53891
W20260620 20:12:50.825155 25466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:50.825438 25466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:50.825476 25466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:50.829622 25466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:50.829687 25466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:50.829746 25466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.130
I20260620 20:12:50.834586 25466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:35323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.91.130
--webserver_port=0
--tserver_master_addrs=127.5.91.190:37373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.130
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:50.835628 25466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:50.836966 25466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:50.846514 25472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:50.846522 25471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:50.846546 25466 server_base.cc:1034] running on GCE node
W20260620 20:12:50.846783 25474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:50.847249 25466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:50.847904 25466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:50.849128 25466 hybrid_clock.cc:648] HybridClock initialized: now 1781986370849087 us; error 57 us; skew 500 ppm
I20260620 20:12:50.851023 25466 webserver.cc:460] Webserver started at http://127.5.91.130:38309/ using document root <none> and password file <none>
I20260620 20:12:50.851596 25466 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:50.851663 25466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:50.851854 25466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:50.854017 25466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-1/data/instance:
uuid: "48c1f2ebd59d4308a4f732b1985af4cf"
format_stamp: "Formatted at 2026-06-20 20:12:50 on dist-test-slave-l8h4"
server_key: "65e8dbef66b3e8125fc95a86d0dd4521"
server_key_iv: "7cd680d21f4e1faca327a03d0471f639"
server_key_version: "encryptionkey@0"
I20260620 20:12:50.854621 25466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance:
uuid: "48c1f2ebd59d4308a4f732b1985af4cf"
format_stamp: "Formatted at 2026-06-20 20:12:50 on dist-test-slave-l8h4"
server_key: "65e8dbef66b3e8125fc95a86d0dd4521"
server_key_iv: "7cd680d21f4e1faca327a03d0471f639"
server_key_version: "encryptionkey@0"
I20260620 20:12:50.858616 25466 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 20:12:50.861150 25480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:50.862439 25466 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:12:50.862550 25466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-1/wal
uuid: "48c1f2ebd59d4308a4f732b1985af4cf"
format_stamp: "Formatted at 2026-06-20 20:12:50 on dist-test-slave-l8h4"
server_key: "65e8dbef66b3e8125fc95a86d0dd4521"
server_key_iv: "7cd680d21f4e1faca327a03d0471f639"
server_key_version: "encryptionkey@0"
I20260620 20:12:50.862653 25466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:50.880537 25466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:50.881237 25466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:50.881412 25466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:50.881990 25466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:50.882997 25466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:12:50.883054 25466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:50.883129 25466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:12:50.883157 25466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:50.891839 25466 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.130:37485
I20260620 20:12:50.891917 25593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.130:37485 every 8 connection(s)
I20260620 20:12:50.892730 25466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-1/data/info.pb
I20260620 20:12:50.899773 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 25466
I20260620 20:12:50.899900 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-1/wal/instance
I20260620 20:12:50.900192 5486 external_mini_cluster.cc:1428] Setting key 4fc2f1c54c99c23875e370acfaf76f0b
I20260620 20:12:50.901976 25314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:50.894499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:46851 (local address 127.5.91.190:37373)
0620 20:12:50.894674 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:50.894679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:50.895669 (+ 990us) server_negotiation.cc:407] Connection header received
0620 20:12:50.896970 (+ 1301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:50.896973 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:50.897022 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:50.897138 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:50.898081 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:50.898692 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:50.900039 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:50.900264 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:50.900361 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:50.901497 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:50.901499 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:50.901501 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:12:50.901512 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:50.901550 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:12:50.901559 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:50.901621 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:50.901778 (+ 157us) server_negotiation.cc:299] Negotiation successful
0620 20:12:50.901840 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260620 20:12:50.902100 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.91.131:0
--local_ip_for_outbound_sockets=127.5.91.131
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:37373
--builtin_ntp_servers=127.5.91.148:35323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 20:12:50.902909 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:50.894884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:37373 (local address 127.5.91.130:46851)
0620 20:12:50.895528 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:50.895572 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:50.896758 (+ 1186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:50.897371 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:50.897378 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:50.897460 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:50.897931 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:50.897942 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:50.898846 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:50.898851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:50.899918 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:50.899930 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:50.900409 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:50.900414 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:50.900605 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:50.901222 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 20:12:50.901223 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:50.901262 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 20:12:50.901325 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:50.901649 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:50.901658 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:50.901662 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 20:12:50.901773 (+ 111us) client_negotiation.cc:240] Negotiation successful
0620 20:12:50.902062 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":128,"threads_started":1}
I20260620 20:12:50.903942 25594 heartbeater.cc:347] Connected to a master server at 127.5.91.190:37373
I20260620 20:12:50.904191 25594 heartbeater.cc:464] Registering TS with master...
I20260620 20:12:50.904706 25594 heartbeater.cc:511] Master 127.5.91.190:37373 requested a full tablet report, sending...
I20260620 20:12:50.905654 25275 ts_manager.cc:194] Registered new tserver with Master: 48c1f2ebd59d4308a4f732b1985af4cf (127.5.91.130:37485)
I20260620 20:12:50.906276 25275 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.91.130:46851
W20260620 20:12:51.012908 25597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:51.013192 25597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:51.013231 25597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:51.017316 25597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:51.017377 25597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:51.017434 25597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.91.131
I20260620 20:12:51.022403 25597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:35323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.91.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.91.131
--webserver_port=0
--tserver_master_addrs=127.5.91.190:37373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.91.131
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:51.023424 25597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:51.024686 25597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:51.033643 25603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:51.033661 25602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:51.033689 25605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:51.033716 25597 server_base.cc:1034] running on GCE node
I20260620 20:12:51.034283 25597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:51.034814 25597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:51.035965 25597 hybrid_clock.cc:648] HybridClock initialized: now 1781986371035923 us; error 52 us; skew 500 ppm
I20260620 20:12:51.037645 25597 webserver.cc:460] Webserver started at http://127.5.91.131:43031/ using document root <none> and password file <none>
I20260620 20:12:51.038198 25597 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:51.038261 25597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:51.038446 25597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:51.040335 25597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-2/data/instance:
uuid: "4e2d90d16ada47649934c8791ef6ab62"
format_stamp: "Formatted at 2026-06-20 20:12:51 on dist-test-slave-l8h4"
server_key: "e43f92d57a645a192cf75c35f738cca7"
server_key_iv: "247a8abf424b82dbc79a4ae1f0a67a41"
server_key_version: "encryptionkey@0"
I20260620 20:12:51.040879 25597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance:
uuid: "4e2d90d16ada47649934c8791ef6ab62"
format_stamp: "Formatted at 2026-06-20 20:12:51 on dist-test-slave-l8h4"
server_key: "e43f92d57a645a192cf75c35f738cca7"
server_key_iv: "247a8abf424b82dbc79a4ae1f0a67a41"
server_key_version: "encryptionkey@0"
I20260620 20:12:51.044342 25597 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 20:12:51.046734 25611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:51.047967 25597 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:12:51.048062 25597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-2/wal
uuid: "4e2d90d16ada47649934c8791ef6ab62"
format_stamp: "Formatted at 2026-06-20 20:12:51 on dist-test-slave-l8h4"
server_key: "e43f92d57a645a192cf75c35f738cca7"
server_key_iv: "247a8abf424b82dbc79a4ae1f0a67a41"
server_key_version: "encryptionkey@0"
I20260620 20:12:51.048175 25597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:51.066505 25597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:51.067169 25597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:51.067329 25597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:51.067884 25597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 20:12:51.068763 25597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 20:12:51.068807 25597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:51.068845 25597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 20:12:51.068876 25597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:51.076840 25597 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.131:46811
I20260620 20:12:51.076871 25724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.131:46811 every 8 connection(s)
I20260620 20:12:51.077677 25597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-2/data/info.pb
I20260620 20:12:51.079320 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 25597
I20260620 20:12:51.079414 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-2/wal/instance
I20260620 20:12:51.079715 5486 external_mini_cluster.cc:1428] Setting key ce15b8ff504e703306dd761fdd12e68d
I20260620 20:12:51.086867 25314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:51.079504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.131:36519 (local address 127.5.91.190:37373)
0620 20:12:51.079684 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:51.079689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:51.080696 (+ 1007us) server_negotiation.cc:407] Connection header received
0620 20:12:51.082279 (+ 1583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:51.082282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:51.082327 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:51.082398 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:51.083248 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.083781 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.085012 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.085259 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.085348 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:51.086390 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:51.086393 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:51.086396 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:12:51.086409 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:51.086431 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:12:51.086439 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:51.086524 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:51.086657 (+ 133us) server_negotiation.cc:299] Negotiation successful
0620 20:12:51.086719 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 20:12:51.087671 25727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:51.079863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:37373 (local address 127.5.91.131:36519)
0620 20:12:51.080548 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:51.080600 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:51.082043 (+ 1443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:51.082524 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:51.082531 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:51.082612 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:12:51.083088 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:51.083100 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.083903 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.083906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:51.084892 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:51.084903 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.085397 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.085401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:51.085553 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:51.086101 (+ 548us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 20:12:51.086103 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:12:51.086173 (+ 70us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 20:12:51.086259 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:12:51.086526 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:12:51.086535 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:12:51.086541 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 20:12:51.086651 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 20:12:51.086930 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":125,"threads_started":1}
I20260620 20:12:51.088690 25725 heartbeater.cc:347] Connected to a master server at 127.5.91.190:37373
I20260620 20:12:51.088883 25725 heartbeater.cc:464] Registering TS with master...
I20260620 20:12:51.089332 25725 heartbeater.cc:511] Master 127.5.91.190:37373 requested a full tablet report, sending...
I20260620 20:12:51.090287 25275 ts_manager.cc:194] Registered new tserver with Master: 4e2d90d16ada47649934c8791ef6ab62 (127.5.91.131:46811)
I20260620 20:12:51.090816 25275 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.91.131:36519
I20260620 20:12:51.093317 5486 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 20:12:51.098840 25314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:51.095191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43602 (local address 127.5.91.190:37373)
0620 20:12:51.095298 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:51.095302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:51.095515 (+ 213us) server_negotiation.cc:407] Connection header received
0620 20:12:51.095612 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:51.095614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:51.095654 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:51.095716 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:51.095955 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.096456 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.097308 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.097535 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.097622 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:51.098387 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:51.098390 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:51.098392 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:12:51.098403 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:51.098420 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:12:51.098428 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:51.098483 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:51.098616 (+ 133us) server_negotiation.cc:299] Negotiation successful
0620 20:12:51.098681 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 20:12:51.101667 25275 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43602:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 20:12:51.104676 25275 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 20:12:51.118918 25739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:51.113564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40244 (local address 127.5.91.130:37485)
0620 20:12:51.114026 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:51.114038 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:51.114482 (+ 444us) server_negotiation.cc:407] Connection header received
0620 20:12:51.114794 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:51.114799 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:51.114948 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:51.115099 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:51.115364 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.116182 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.117590 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.118253 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.118350 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:51.118551 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:51.118661 (+ 110us) server_negotiation.cc:299] Negotiation successful
0620 20:12:51.118757 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":138,"threads_started":1}
I20260620 20:12:51.118916 25738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:51.113356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:37485 (local address 127.0.0.1:40244)
0620 20:12:51.114287 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:51.114342 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:51.114562 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:51.115109 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:51.115115 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:51.115137 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:51.115258 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:51.115264 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.116290 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.116293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:51.117478 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:51.117487 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.118365 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.118369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:51.118488 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:51.118526 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 20:12:51.118633 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 20:12:51.118757 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"spinlock_wait_cycles":247168,"thread_start_us":76,"threads_started":1}
I20260620 20:12:51.120117 25736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:51.113207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:40683 (local address 127.0.0.1:46056)
0620 20:12:51.114287 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:51.114342 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:51.114491 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:51.115239 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:51.115241 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:51.115255 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:51.115416 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:51.115422 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.116939 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.116942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:51.118310 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:51.118317 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.119704 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.119708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:51.119799 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:51.119824 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 20:12:51.119898 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 20:12:51.119968 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":911,"mutex_wait_us":175,"spinlock_wait_cycles":9088,"thread_start_us":179,"threads_started":1}
I20260620 20:12:51.120354 25740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:51.113669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46056 (local address 127.5.91.129:40683)
0620 20:12:51.114086 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:51.114096 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:51.114748 (+ 652us) server_negotiation.cc:407] Connection header received
0620 20:12:51.114825 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:51.114828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:51.114966 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:51.115099 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:51.115517 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.116802 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.118431 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.119578 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.119681 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:51.119843 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:51.119926 (+ 83us) server_negotiation.cc:299] Negotiation successful
0620 20:12:51.120019 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":86,"threads_started":1}
I20260620 20:12:51.121922 25528 tablet_service.cc:1467] Processing CreateTablet for tablet e6dfcddc04044f1cbe309f35432fe33f (DEFAULT_TABLE table=test-table [id=9bb1a296fe2d411a9c914efe53e961b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:12:51.122526 25741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:51.113669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46156 (local address 127.5.91.131:46811)
0620 20:12:51.114052 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:51.114062 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:51.114544 (+ 482us) server_negotiation.cc:407] Connection header received
0620 20:12:51.114732 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:51.114737 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:51.114949 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:51.115099 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:51.115555 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.116803 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.118431 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.119201 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.119294 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:51.119461 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:51.122257 (+ 2796us) server_negotiation.cc:299] Negotiation successful
0620 20:12:51.122362 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":99,"threads_started":1}
I20260620 20:12:51.122867 25737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:51.113003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:46811 (local address 127.0.0.1:46156)
0620 20:12:51.114330 (+ 1327us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:51.114344 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:51.114584 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:51.115258 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:51.115261 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:51.115279 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:51.115452 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:51.115459 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.116939 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.116942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:51.118294 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:51.118305 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.119525 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.119528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:51.119625 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:51.119650 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 20:12:51.119710 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 20:12:51.119783 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"mutex_wait_us":110,"spinlock_wait_cycles":1280,"thread_start_us":86,"threads_started":1}
I20260620 20:12:51.122937 25528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6dfcddc04044f1cbe309f35432fe33f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:51.122742 25397 tablet_service.cc:1467] Processing CreateTablet for tablet e6dfcddc04044f1cbe309f35432fe33f (DEFAULT_TABLE table=test-table [id=9bb1a296fe2d411a9c914efe53e961b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:12:51.123708 25397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6dfcddc04044f1cbe309f35432fe33f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:51.125437 25659 tablet_service.cc:1467] Processing CreateTablet for tablet e6dfcddc04044f1cbe309f35432fe33f (DEFAULT_TABLE table=test-table [id=9bb1a296fe2d411a9c914efe53e961b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 20:12:51.126534 25659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6dfcddc04044f1cbe309f35432fe33f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:51.129516 25742 tablet_bootstrap.cc:492] T e6dfcddc04044f1cbe309f35432fe33f P 34eeb5a72d574c72b27b17c0befe5ff9: Bootstrap starting.
I20260620 20:12:51.131717 25742 tablet_bootstrap.cc:654] T e6dfcddc04044f1cbe309f35432fe33f P 34eeb5a72d574c72b27b17c0befe5ff9: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:51.132404 25742 log.cc:824] T e6dfcddc04044f1cbe309f35432fe33f P 34eeb5a72d574c72b27b17c0befe5ff9: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:51.133133 25745 tablet_bootstrap.cc:492] T e6dfcddc04044f1cbe309f35432fe33f P 4e2d90d16ada47649934c8791ef6ab62: Bootstrap starting.
I20260620 20:12:51.133742 25743 tablet_bootstrap.cc:492] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf: Bootstrap starting.
I20260620 20:12:51.134418 25742 tablet_bootstrap.cc:492] T e6dfcddc04044f1cbe309f35432fe33f P 34eeb5a72d574c72b27b17c0befe5ff9: No bootstrap required, opened a new log
I20260620 20:12:51.134630 25742 ts_tablet_manager.cc:1397] T e6dfcddc04044f1cbe309f35432fe33f P 34eeb5a72d574c72b27b17c0befe5ff9: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 20:12:51.134917 25745 tablet_bootstrap.cc:654] T e6dfcddc04044f1cbe309f35432fe33f P 4e2d90d16ada47649934c8791ef6ab62: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:51.135536 25745 log.cc:824] T e6dfcddc04044f1cbe309f35432fe33f P 4e2d90d16ada47649934c8791ef6ab62: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:51.135715 25743 tablet_bootstrap.cc:654] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:51.136468 25743 log.cc:824] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:51.137234 25745 tablet_bootstrap.cc:492] T e6dfcddc04044f1cbe309f35432fe33f P 4e2d90d16ada47649934c8791ef6ab62: No bootstrap required, opened a new log
I20260620 20:12:51.137408 25745 ts_tablet_manager.cc:1397] T e6dfcddc04044f1cbe309f35432fe33f P 4e2d90d16ada47649934c8791ef6ab62: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260620 20:12:51.138238 25743 tablet_bootstrap.cc:492] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf: No bootstrap required, opened a new log
I20260620 20:12:51.138408 25743 ts_tablet_manager.cc:1397] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 20:12:51.138779 25742 raft_consensus.cc:348] T e6dfcddc04044f1cbe309f35432fe33f P 34eeb5a72d574c72b27b17c0befe5ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2d90d16ada47649934c8791ef6ab62" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46811 } } peers { permanent_uuid: "34eeb5a72d574c72b27b17c0befe5ff9" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40683 } } peers { permanent_uuid: "48c1f2ebd59d4308a4f732b1985af4cf" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 37485 } }
I20260620 20:12:51.139047 25742 raft_consensus.cc:374] T e6dfcddc04044f1cbe309f35432fe33f P 34eeb5a72d574c72b27b17c0befe5ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:51.139112 25742 raft_consensus.cc:729] T e6dfcddc04044f1cbe309f35432fe33f P 34eeb5a72d574c72b27b17c0befe5ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34eeb5a72d574c72b27b17c0befe5ff9, State: Initialized, Role: FOLLOWER
I20260620 20:12:51.139544 25742 consensus_queue.cc:260] T e6dfcddc04044f1cbe309f35432fe33f P 34eeb5a72d574c72b27b17c0befe5ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2d90d16ada47649934c8791ef6ab62" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46811 } } peers { permanent_uuid: "34eeb5a72d574c72b27b17c0befe5ff9" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40683 } } peers { permanent_uuid: "48c1f2ebd59d4308a4f732b1985af4cf" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 37485 } }
I20260620 20:12:51.140223 25463 heartbeater.cc:503] Master 127.5.91.190:37373 was elected leader, sending a full tablet report...
I20260620 20:12:51.140501 25742 ts_tablet_manager.cc:1428] T e6dfcddc04044f1cbe309f35432fe33f P 34eeb5a72d574c72b27b17c0befe5ff9: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260620 20:12:51.141834 25745 raft_consensus.cc:348] T e6dfcddc04044f1cbe309f35432fe33f P 4e2d90d16ada47649934c8791ef6ab62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2d90d16ada47649934c8791ef6ab62" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46811 } } peers { permanent_uuid: "34eeb5a72d574c72b27b17c0befe5ff9" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40683 } } peers { permanent_uuid: "48c1f2ebd59d4308a4f732b1985af4cf" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 37485 } }
I20260620 20:12:51.142026 25745 raft_consensus.cc:374] T e6dfcddc04044f1cbe309f35432fe33f P 4e2d90d16ada47649934c8791ef6ab62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:51.142077 25745 raft_consensus.cc:729] T e6dfcddc04044f1cbe309f35432fe33f P 4e2d90d16ada47649934c8791ef6ab62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e2d90d16ada47649934c8791ef6ab62, State: Initialized, Role: FOLLOWER
I20260620 20:12:51.142041 25743 raft_consensus.cc:348] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2d90d16ada47649934c8791ef6ab62" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46811 } } peers { permanent_uuid: "34eeb5a72d574c72b27b17c0befe5ff9" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40683 } } peers { permanent_uuid: "48c1f2ebd59d4308a4f732b1985af4cf" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 37485 } }
I20260620 20:12:51.142295 25743 raft_consensus.cc:374] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:51.142339 25743 raft_consensus.cc:729] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48c1f2ebd59d4308a4f732b1985af4cf, State: Initialized, Role: FOLLOWER
I20260620 20:12:51.142488 25745 consensus_queue.cc:260] T e6dfcddc04044f1cbe309f35432fe33f P 4e2d90d16ada47649934c8791ef6ab62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2d90d16ada47649934c8791ef6ab62" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46811 } } peers { permanent_uuid: "34eeb5a72d574c72b27b17c0befe5ff9" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40683 } } peers { permanent_uuid: "48c1f2ebd59d4308a4f732b1985af4cf" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 37485 } }
I20260620 20:12:51.142701 25743 consensus_queue.cc:260] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2d90d16ada47649934c8791ef6ab62" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46811 } } peers { permanent_uuid: "34eeb5a72d574c72b27b17c0befe5ff9" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40683 } } peers { permanent_uuid: "48c1f2ebd59d4308a4f732b1985af4cf" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 37485 } }
I20260620 20:12:51.143154 25725 heartbeater.cc:503] Master 127.5.91.190:37373 was elected leader, sending a full tablet report...
I20260620 20:12:51.143376 25745 ts_tablet_manager.cc:1428] T e6dfcddc04044f1cbe309f35432fe33f P 4e2d90d16ada47649934c8791ef6ab62: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260620 20:12:51.143570 25594 heartbeater.cc:503] Master 127.5.91.190:37373 was elected leader, sending a full tablet report...
I20260620 20:12:51.145311 25743 ts_tablet_manager.cc:1428] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
W20260620 20:12:51.204367 25464 tablet.cc:2369] T e6dfcddc04044f1cbe309f35432fe33f P 34eeb5a72d574c72b27b17c0befe5ff9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 20:12:51.328715 25726 tablet.cc:2369] T e6dfcddc04044f1cbe309f35432fe33f P 4e2d90d16ada47649934c8791ef6ab62: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:12:51.331653 25750 raft_consensus.cc:482] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 20:12:51.331851 25750 raft_consensus.cc:504] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2d90d16ada47649934c8791ef6ab62" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46811 } } peers { permanent_uuid: "34eeb5a72d574c72b27b17c0befe5ff9" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40683 } } peers { permanent_uuid: "48c1f2ebd59d4308a4f732b1985af4cf" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 37485 } }
I20260620 20:12:51.332968 25750 leader_election.cc:288] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e2d90d16ada47649934c8791ef6ab62 (127.5.91.131:46811), 34eeb5a72d574c72b27b17c0befe5ff9 (127.5.91.129:40683)
I20260620 20:12:51.336774 25741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:51.333390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:38153 (local address 127.5.91.131:46811)
0620 20:12:51.333530 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:51.333534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:51.333813 (+ 279us) server_negotiation.cc:407] Connection header received
0620 20:12:51.333879 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:51.333882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:51.333926 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:51.334012 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:51.334338 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.334871 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.335908 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.336301 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.336411 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:51.336457 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:51.336605 (+ 148us) server_negotiation.cc:299] Negotiation successful
0620 20:12:51.336676 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 20:12:51.336762 25740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:51.333390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.91.130:41135 (local address 127.5.91.129:40683)
0620 20:12:51.333508 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:51.333513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:51.333532 (+ 19us) server_negotiation.cc:407] Connection header received
0620 20:12:51.333591 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:51.333594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:51.333646 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:51.333747 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 20:12:51.334175 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.334743 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.335818 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.336231 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.336323 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:51.336366 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:51.336561 (+ 195us) server_negotiation.cc:299] Negotiation successful
0620 20:12:51.336642 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 20:12:51.336762 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:51.333255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:40683 (local address 127.5.91.130:41135)
0620 20:12:51.333363 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:51.333378 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:51.333472 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:51.333868 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:51.333871 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:51.333893 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:51.334022 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:51.334030 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.334853 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.334856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:51.335688 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:51.335698 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.336346 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.336350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:51.336442 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:51.336465 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 20:12:51.336530 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 20:12:51.336621 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260620 20:12:51.336781 25751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:51.333255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.131:46811 (local address 127.5.91.130:38153)
0620 20:12:51.333685 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:51.333698 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:12:51.333792 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:12:51.334061 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:12:51.334063 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:12:51.334084 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 20:12:51.334229 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:51.334236 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.334990 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.334993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:51.335787 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:12:51.335798 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.336413 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.336416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:12:51.336484 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:51.336500 (+ 16us) client_negotiation.cc:769] Sending connection context
0620 20:12:51.336577 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 20:12:51.336638 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":110,"thread_start_us":129,"threads_started":1}
I20260620 20:12:51.337419 25417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6dfcddc04044f1cbe309f35432fe33f" candidate_uuid: "48c1f2ebd59d4308a4f732b1985af4cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34eeb5a72d574c72b27b17c0befe5ff9" is_pre_election: true
I20260620 20:12:51.337424 25679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6dfcddc04044f1cbe309f35432fe33f" candidate_uuid: "48c1f2ebd59d4308a4f732b1985af4cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e2d90d16ada47649934c8791ef6ab62" is_pre_election: true
I20260620 20:12:51.337682 25417 raft_consensus.cc:2454] T e6dfcddc04044f1cbe309f35432fe33f P 34eeb5a72d574c72b27b17c0befe5ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48c1f2ebd59d4308a4f732b1985af4cf in term 0.
I20260620 20:12:51.337688 25679 raft_consensus.cc:2454] T e6dfcddc04044f1cbe309f35432fe33f P 4e2d90d16ada47649934c8791ef6ab62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48c1f2ebd59d4308a4f732b1985af4cf in term 0.
I20260620 20:12:51.338161 25483 leader_election.cc:302] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34eeb5a72d574c72b27b17c0befe5ff9, 48c1f2ebd59d4308a4f732b1985af4cf; no voters:
I20260620 20:12:51.338402 25750 raft_consensus.cc:2781] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 20:12:51.338483 25750 raft_consensus.cc:482] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 20:12:51.338531 25750 raft_consensus.cc:3037] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:51.339517 25750 raft_consensus.cc:504] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2d90d16ada47649934c8791ef6ab62" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46811 } } peers { permanent_uuid: "34eeb5a72d574c72b27b17c0befe5ff9" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40683 } } peers { permanent_uuid: "48c1f2ebd59d4308a4f732b1985af4cf" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 37485 } }
I20260620 20:12:51.339835 25750 leader_election.cc:288] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf [CANDIDATE]: Term 1 election: Requested vote from peers 4e2d90d16ada47649934c8791ef6ab62 (127.5.91.131:46811), 34eeb5a72d574c72b27b17c0befe5ff9 (127.5.91.129:40683)
I20260620 20:12:51.340193 25679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6dfcddc04044f1cbe309f35432fe33f" candidate_uuid: "48c1f2ebd59d4308a4f732b1985af4cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e2d90d16ada47649934c8791ef6ab62"
I20260620 20:12:51.340224 25417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6dfcddc04044f1cbe309f35432fe33f" candidate_uuid: "48c1f2ebd59d4308a4f732b1985af4cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34eeb5a72d574c72b27b17c0befe5ff9"
I20260620 20:12:51.340333 25679 raft_consensus.cc:3037] T e6dfcddc04044f1cbe309f35432fe33f P 4e2d90d16ada47649934c8791ef6ab62 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:51.340333 25417 raft_consensus.cc:3037] T e6dfcddc04044f1cbe309f35432fe33f P 34eeb5a72d574c72b27b17c0befe5ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:51.341694 25679 raft_consensus.cc:2454] T e6dfcddc04044f1cbe309f35432fe33f P 4e2d90d16ada47649934c8791ef6ab62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48c1f2ebd59d4308a4f732b1985af4cf in term 1.
I20260620 20:12:51.341694 25417 raft_consensus.cc:2454] T e6dfcddc04044f1cbe309f35432fe33f P 34eeb5a72d574c72b27b17c0befe5ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48c1f2ebd59d4308a4f732b1985af4cf in term 1.
I20260620 20:12:51.342006 25482 leader_election.cc:302] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48c1f2ebd59d4308a4f732b1985af4cf, 4e2d90d16ada47649934c8791ef6ab62; no voters:
I20260620 20:12:51.342190 25750 raft_consensus.cc:2781] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:12:51.342414 25750 raft_consensus.cc:686] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf [term 1 LEADER]: Becoming Leader. State: Replica: 48c1f2ebd59d4308a4f732b1985af4cf, State: Running, Role: LEADER
I20260620 20:12:51.342721 25750 consensus_queue.cc:237] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2d90d16ada47649934c8791ef6ab62" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46811 } } peers { permanent_uuid: "34eeb5a72d574c72b27b17c0befe5ff9" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40683 } } peers { permanent_uuid: "48c1f2ebd59d4308a4f732b1985af4cf" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 37485 } }
I20260620 20:12:51.346505 25273 catalog_manager.cc:5495] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 48c1f2ebd59d4308a4f732b1985af4cf (127.5.91.130). New cstate: current_term: 1 leader_uuid: "48c1f2ebd59d4308a4f732b1985af4cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2d90d16ada47649934c8791ef6ab62" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34eeb5a72d574c72b27b17c0befe5ff9" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48c1f2ebd59d4308a4f732b1985af4cf" member_type: VOTER last_known_addr { host: "127.5.91.130" port: 37485 } health_report { overall_health: HEALTHY } } }
W20260620 20:12:51.394289 25595 tablet.cc:2369] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:12:51.413308 25739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:51.409703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40252 (local address 127.5.91.130:37485)
0620 20:12:51.409846 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:51.409850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:51.409863 (+ 13us) server_negotiation.cc:407] Connection header received
0620 20:12:51.409941 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:51.409943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:51.410033 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:51.410126 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:12:51.410580 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.411081 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.411982 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:51.412194 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:51.412274 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:51.412542 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:12:51.412954 (+ 412us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:12:51.413037 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:51.413084 (+ 47us) server_negotiation.cc:299] Negotiation successful
0620 20:12:51.413131 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260620 20:12:51.418977 25679 raft_consensus.cc:1261] T e6dfcddc04044f1cbe309f35432fe33f P 4e2d90d16ada47649934c8791ef6ab62 [term 1 FOLLOWER]: Refusing update from remote peer 48c1f2ebd59d4308a4f732b1985af4cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:12:51.418994 25417 raft_consensus.cc:1261] T e6dfcddc04044f1cbe309f35432fe33f P 34eeb5a72d574c72b27b17c0befe5ff9 [term 1 FOLLOWER]: Refusing update from remote peer 48c1f2ebd59d4308a4f732b1985af4cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:12:51.419885 25752 consensus_queue.cc:1035] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e2d90d16ada47649934c8791ef6ab62" member_type: VOTER last_known_addr { host: "127.5.91.131" port: 46811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:12:51.420095 25750 consensus_queue.cc:1035] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "34eeb5a72d574c72b27b17c0befe5ff9" member_type: VOTER last_known_addr { host: "127.5.91.129" port: 40683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:12:51.427285 25756 mvcc.cc:205] Tried to move back new op lower bound from 7299016177324462080 to 7299016177023066112. Current Snapshot: MvccSnapshot[applied={T|T < 7299016177324462080}]
I20260620 20:12:51.427490 25755 mvcc.cc:205] Tried to move back new op lower bound from 7299016177324462080 to 7299016177023066112. Current Snapshot: MvccSnapshot[applied={T|T < 7299016177324462080}]
I20260620 20:12:51.436714 25273 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:12:51.436882 25273 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 20:12:51.439369 25273 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 418b2e3cd8744198b67394614029b898: Sending DeleteTablet for 3 replicas of tablet e6dfcddc04044f1cbe309f35432fe33f
I20260620 20:12:51.440235 25659 tablet_service.cc:1514] Processing DeleteTablet for tablet e6dfcddc04044f1cbe309f35432fe33f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:12:51 UTC) from {username='slave'} at 127.0.0.1:46156
I20260620 20:12:51.440232 25397 tablet_service.cc:1514] Processing DeleteTablet for tablet e6dfcddc04044f1cbe309f35432fe33f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:12:51 UTC) from {username='slave'} at 127.0.0.1:46056
I20260620 20:12:51.441054 25766 tablet_replica.cc:330] T e6dfcddc04044f1cbe309f35432fe33f P 34eeb5a72d574c72b27b17c0befe5ff9: stopping tablet replica
I20260620 20:12:51.441073 25528 tablet_service.cc:1514] Processing DeleteTablet for tablet e6dfcddc04044f1cbe309f35432fe33f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 20:12:51 UTC) from {username='slave'} at 127.0.0.1:40244
I20260620 20:12:51.441396 25766 raft_consensus.cc:2229] T e6dfcddc04044f1cbe309f35432fe33f P 34eeb5a72d574c72b27b17c0befe5ff9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:12:51.441553 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 25335
I20260620 20:12:51.441756 25766 raft_consensus.cc:2258] T e6dfcddc04044f1cbe309f35432fe33f P 34eeb5a72d574c72b27b17c0befe5ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:12:51.442164 25765 tablet_replica.cc:330] T e6dfcddc04044f1cbe309f35432fe33f P 4e2d90d16ada47649934c8791ef6ab62: stopping tablet replica
I20260620 20:12:51.442449 25765 raft_consensus.cc:2229] T e6dfcddc04044f1cbe309f35432fe33f P 4e2d90d16ada47649934c8791ef6ab62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 20:12:51.442792 25765 raft_consensus.cc:2258] T e6dfcddc04044f1cbe309f35432fe33f P 4e2d90d16ada47649934c8791ef6ab62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:12:51.444446 25765 ts_tablet_manager.cc:1905] T e6dfcddc04044f1cbe309f35432fe33f P 4e2d90d16ada47649934c8791ef6ab62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 20:12:51.447203 25765 ts_tablet_manager.cc:1918] T e6dfcddc04044f1cbe309f35432fe33f P 4e2d90d16ada47649934c8791ef6ab62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 20:12:51.447302 25765 log.cc:1196] T e6dfcddc04044f1cbe309f35432fe33f P 4e2d90d16ada47649934c8791ef6ab62: Deleting WAL directory at /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781986111135528-5486-0/minicluster-data/ts-2/wal/wals/e6dfcddc04044f1cbe309f35432fe33f
I20260620 20:12:51.447716 25765 ts_tablet_manager.cc:1939] T e6dfcddc04044f1cbe309f35432fe33f P 4e2d90d16ada47649934c8791ef6ab62: Deleting consensus metadata
I20260620 20:12:51.449281 25260 catalog_manager.cc:4841] TS 4e2d90d16ada47649934c8791ef6ab62 (127.5.91.131:46811): tablet e6dfcddc04044f1cbe309f35432fe33f (table test-table [id=9bb1a296fe2d411a9c914efe53e961b2]) successfully deleted
I20260620 20:12:51.449481 25767 tablet_replica.cc:330] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf: stopping tablet replica
I20260620 20:12:51.449929 25767 raft_consensus.cc:2229] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf [term 1 LEADER]: Raft consensus shutting down.
W20260620 20:12:51.450194 25261 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.91.129:40683 (error 108)
I20260620 20:12:51.450394 25767 raft_consensus.cc:2258] T e6dfcddc04044f1cbe309f35432fe33f P 48c1f2ebd59d4308a4f732b1985af4cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 20:12:51.451599 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 25466
I20260620 20:12:51.452206 25737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:51.451798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.129:40683 (local address 127.0.0.1:46064)
0620 20:12:51.452019 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:51.452112 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.129:40683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260620 20:12:51.452399 25261 catalog_manager.cc:4584] TS 34eeb5a72d574c72b27b17c0befe5ff9 (127.5.91.129:40683): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e6dfcddc04044f1cbe309f35432fe33f: Network error: Client connection negotiation failed: client connection to 127.5.91.129:40683: connect: Connection refused (error 111)
I20260620 20:12:51.456923 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 25597
I20260620 20:12:51.457855 25737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:51.457265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.130:37485 (local address 127.0.0.1:40264)
0620 20:12:51.457714 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0620 20:12:51.457770 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.130:37485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377}
I20260620 20:12:51.461797 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 25244
2026-06-20T20:12:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1195 ms)
[----------] 6 tests from EncryptionPolicyTest (10236 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:12:51 dist-test-slave-l8h4 krb5kdc[25771](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:12:51 dist-test-slave-l8h4 krb5kdc[25771](info): set up 2 sockets
Jun 20 20:12:51 dist-test-slave-l8h4 krb5kdc[25771](info): commencing operation
krb5kdc: starting...
W20260620 20:12:52.438870 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:12:52 dist-test-slave-l8h4 krb5kdc[25771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:12:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:12:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 20:12:52.557372 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45245
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:12:52.663534 25784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:52.663793 25784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:52.663834 25784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:52.667747 25784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:12:52.667796 25784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:52.667814 25784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:52.667829 25784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:12:52.667842 25784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:12:52.673038 25784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45245
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:52.674031 25784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:52.675576 25784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:52.684566 25789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:52.684646 25790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:52.684564 25792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:52.685084 25784 server_base.cc:1034] running on GCE node
I20260620 20:12:52.685884 25784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:52.687553 25784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:52.688875 25784 hybrid_clock.cc:648] HybridClock initialized: now 1781986372688825 us; error 52 us; skew 500 ppm
Jun 20 20:12:52 dist-test-slave-l8h4 krb5kdc[25771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986372, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:52.692497 25784 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 20:12:52.693454 25784 webserver.cc:460] Webserver started at http://127.0.0.1:36437/ using document root <none> and password file <none>
I20260620 20:12:52.694020 25784 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:52.694082 25784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:52.694305 25784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:52.696331 25784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "3db82c7e055641e685ae3284357e104a"
format_stamp: "Formatted at 2026-06-20 20:12:52 on dist-test-slave-l8h4"
server_key: "b5d7ef3e56628e6ff256a984a51458b1"
server_key_iv: "c79f5e70769609a248ec5206b18fe3d9"
server_key_version: "encryptionkey@0"
I20260620 20:12:52.696895 25784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "3db82c7e055641e685ae3284357e104a"
format_stamp: "Formatted at 2026-06-20 20:12:52 on dist-test-slave-l8h4"
server_key: "b5d7ef3e56628e6ff256a984a51458b1"
server_key_iv: "c79f5e70769609a248ec5206b18fe3d9"
server_key_version: "encryptionkey@0"
I20260620 20:12:52.700771 25784 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260620 20:12:52.703277 25799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:52.704468 25784 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:12:52.704607 25784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "3db82c7e055641e685ae3284357e104a"
format_stamp: "Formatted at 2026-06-20 20:12:52 on dist-test-slave-l8h4"
server_key: "b5d7ef3e56628e6ff256a984a51458b1"
server_key_iv: "c79f5e70769609a248ec5206b18fe3d9"
server_key_version: "encryptionkey@0"
I20260620 20:12:52.704723 25784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:52.729530 25784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:52.732645 25784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:52.732802 25784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:52.739322 25784 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45245
I20260620 20:12:52.739375 25851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45245 every 8 connection(s)
I20260620 20:12:52.740231 25784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:12:52.742918 25852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:52.743847 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 25784
I20260620 20:12:52.743965 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:12:52.744261 5486 external_mini_cluster.cc:1428] Setting key 9ffdc5147c48a445d87c83ae8f3e729b
I20260620 20:12:52.748489 25852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3db82c7e055641e685ae3284357e104a: Bootstrap starting.
I20260620 20:12:52.750849 25852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3db82c7e055641e685ae3284357e104a: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:52.751538 25852 log.cc:824] T 00000000000000000000000000000000 P 3db82c7e055641e685ae3284357e104a: Log is configured to *not* fsync() on all Append() calls
Jun 20 20:12:52 dist-test-slave-l8h4 krb5kdc[25771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 20:12:52.753393 25852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3db82c7e055641e685ae3284357e104a: No bootstrap required, opened a new log
I20260620 20:12:52.756830 25852 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3db82c7e055641e685ae3284357e104a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3db82c7e055641e685ae3284357e104a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45245 } }
I20260620 20:12:52.757048 25852 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3db82c7e055641e685ae3284357e104a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:52.757082 25852 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3db82c7e055641e685ae3284357e104a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3db82c7e055641e685ae3284357e104a, State: Initialized, Role: FOLLOWER
I20260620 20:12:52.757416 25855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:52.745529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34439 (local address 127.0.0.1:45245)
0620 20:12:52.745943 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:52.745958 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:52.745991 (+ 33us) server_negotiation.cc:407] Connection header received
0620 20:12:52.746690 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:52.746706 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:52.747049 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:52.747340 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:52.748069 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:52.749035 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:52.750165 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:52.750642 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:52.750785 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:52.753150 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:52.753155 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:52.753168 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:52.753194 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:52.754728 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:52.755194 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:52.755199 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:52.755203 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:52.755305 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:52.755610 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:52.755613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:52.755614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:52.755981 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:52.756176 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:52.756497 (+ 321us) server_negotiation.cc:299] Negotiation successful
0620 20:12:52.756693 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":125,"threads_started":1}
I20260620 20:12:52.757483 25852 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3db82c7e055641e685ae3284357e104a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3db82c7e055641e685ae3284357e104a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45245 } }
I20260620 20:12:52.757627 25852 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3db82c7e055641e685ae3284357e104a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:12:52.757688 25852 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3db82c7e055641e685ae3284357e104a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:12:52.757778 25852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3db82c7e055641e685ae3284357e104a [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:52.758677 25852 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3db82c7e055641e685ae3284357e104a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3db82c7e055641e685ae3284357e104a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45245 } }
I20260620 20:12:52.758984 25852 leader_election.cc:302] T 00000000000000000000000000000000 P 3db82c7e055641e685ae3284357e104a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3db82c7e055641e685ae3284357e104a; no voters:
I20260620 20:12:52.759267 25852 leader_election.cc:288] T 00000000000000000000000000000000 P 3db82c7e055641e685ae3284357e104a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:12:52.759342 25857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3db82c7e055641e685ae3284357e104a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:12:52.759744 25857 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3db82c7e055641e685ae3284357e104a [term 1 LEADER]: Becoming Leader. State: Replica: 3db82c7e055641e685ae3284357e104a, State: Running, Role: LEADER
I20260620 20:12:52.760015 25857 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3db82c7e055641e685ae3284357e104a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3db82c7e055641e685ae3284357e104a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45245 } }
I20260620 20:12:52.760138 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:12:52.760468 25852 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3db82c7e055641e685ae3284357e104a [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:12:52.761636 25858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3db82c7e055641e685ae3284357e104a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3db82c7e055641e685ae3284357e104a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3db82c7e055641e685ae3284357e104a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45245 } } }
I20260620 20:12:52.761740 25858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3db82c7e055641e685ae3284357e104a [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:52.761963 25859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3db82c7e055641e685ae3284357e104a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3db82c7e055641e685ae3284357e104a. Latest consensus state: current_term: 1 leader_uuid: "3db82c7e055641e685ae3284357e104a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3db82c7e055641e685ae3284357e104a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45245 } } }
I20260620 20:12:52.762055 25859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3db82c7e055641e685ae3284357e104a [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:52.762355 25871 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:12:52.765265 25871 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:12:52.770835 25855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:52.762436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54527 (local address 127.0.0.1:45245)
0620 20:12:52.762593 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:52.762597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:52.763131 (+ 534us) server_negotiation.cc:407] Connection header received
0620 20:12:52.763258 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:52.763261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:52.763302 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:52.763366 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:52.764037 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:52.764586 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:52.765389 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:52.765611 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:52.765695 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:52.766936 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:52.766940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:52.766943 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:52.766976 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:52.769024 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:52.769554 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:52.769557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:52.769559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:52.769637 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:52.769960 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:52.769963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:52.769964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:52.770193 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:52.770334 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:52.770603 (+ 269us) server_negotiation.cc:299] Negotiation successful
0620 20:12:52.770695 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260620 20:12:52.771261 25871 catalog_manager.cc:1347] Generated new cluster ID: ea14f24a5ab94da9831deeb1cad9d021
I20260620 20:12:52.771353 25871 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:12:52.786504 25871 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:12:52.787473 25871 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:12:52.791561 25871 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3db82c7e055641e685ae3284357e104a: Generated new TSK 0
I20260620 20:12:52.792223 25871 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:12:52.798208 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 25784
W20260620 20:12:52.802889 25769 connection.cc:535] client connection to 127.0.0.1:45245 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45245): Connection reset by peer (error 104)
2026-06-20T20:12:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1334 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:12:52 dist-test-slave-l8h4 krb5kdc[25883](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:12:52 dist-test-slave-l8h4 krb5kdc[25883](info): set up 2 sockets
Jun 20 20:12:52 dist-test-slave-l8h4 krb5kdc[25883](info): commencing operation
krb5kdc: starting...
W20260620 20:12:53.767192 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.951s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:12:53 dist-test-slave-l8h4 krb5kdc[25883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:12:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:12:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 20:12:53.910655 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44299
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:12:54.025309 25896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:54.025597 25896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:54.025647 25896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:54.029917 25896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:12:54.030009 25896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:54.030030 25896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:54.030045 25896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:12:54.030059 25896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:12:54.035423 25896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44299
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:54.036490 25896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:54.037458 25896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:54.045689 25902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:54.045689 25904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:54.045794 25901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:54.045990 25896 server_base.cc:1034] running on GCE node
I20260620 20:12:54.046669 25896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:54.047750 25896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:54.048923 25896 hybrid_clock.cc:648] HybridClock initialized: now 1781986374048908 us; error 25 us; skew 500 ppm
Jun 20 20:12:54 dist-test-slave-l8h4 krb5kdc[25883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986374, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:54.052456 25896 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 20:12:54.053431 25896 webserver.cc:460] Webserver started at http://127.0.0.1:37065/ using document root <none> and password file <none>
I20260620 20:12:54.053968 25896 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:54.054039 25896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:54.054253 25896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:54.056298 25896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "78bf931ba1d048c4beb664022716590f"
format_stamp: "Formatted at 2026-06-20 20:12:54 on dist-test-slave-l8h4"
server_key: "c1bfba738c28fa7de340ca94715b782f"
server_key_iv: "7fc0a01c56a406812a5344250607f1ca"
server_key_version: "encryptionkey@0"
I20260620 20:12:54.056825 25896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "78bf931ba1d048c4beb664022716590f"
format_stamp: "Formatted at 2026-06-20 20:12:54 on dist-test-slave-l8h4"
server_key: "c1bfba738c28fa7de340ca94715b782f"
server_key_iv: "7fc0a01c56a406812a5344250607f1ca"
server_key_version: "encryptionkey@0"
I20260620 20:12:54.060779 25896 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 20:12:54.063446 25911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:54.064755 25896 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 20:12:54.064879 25896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "78bf931ba1d048c4beb664022716590f"
format_stamp: "Formatted at 2026-06-20 20:12:54 on dist-test-slave-l8h4"
server_key: "c1bfba738c28fa7de340ca94715b782f"
server_key_iv: "7fc0a01c56a406812a5344250607f1ca"
server_key_version: "encryptionkey@0"
I20260620 20:12:54.065004 25896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:54.086432 25896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:54.089576 25896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:54.089762 25896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:54.096731 25896 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44299
I20260620 20:12:54.096849 25963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44299 every 8 connection(s)
I20260620 20:12:54.097718 25896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:12:54.100824 25964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:54.106755 25964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f: Bootstrap starting.
I20260620 20:12:54.107844 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 25896
I20260620 20:12:54.107997 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:12:54.108285 5486 external_mini_cluster.cc:1428] Setting key eb959059a602d057c96ae0be5b715205
I20260620 20:12:54.109576 25964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:54.110322 25964 log.cc:824] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:54.112250 25964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f: No bootstrap required, opened a new log
I20260620 20:12:54.116307 25964 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78bf931ba1d048c4beb664022716590f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44299 } }
I20260620 20:12:54.116532 25964 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:54.116586 25964 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78bf931ba1d048c4beb664022716590f, State: Initialized, Role: FOLLOWER
I20260620 20:12:54.117018 25964 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78bf931ba1d048c4beb664022716590f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44299 } }
I20260620 20:12:54.117173 25964 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:12:54.117237 25964 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:12:54.117321 25964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:54.118165 25967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:54.109419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44459 (local address 127.0.0.1:44299)
0620 20:12:54.109838 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:54.109853 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:54.109887 (+ 34us) server_negotiation.cc:407] Connection header received
0620 20:12:54.110675 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:54.110703 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:54.111023 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:54.111399 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:54.112008 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:54.113410 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:54.114663 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:54.115185 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:54.115302 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:54.116338 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:54.116344 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:54.116356 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:12:54.116801 (+ 445us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:54.116857 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:12:54.116882 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:54.116990 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:54.117228 (+ 238us) server_negotiation.cc:299] Negotiation successful
0620 20:12:54.117367 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":113,"threads_started":1}
I20260620 20:12:54.118131 25964 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78bf931ba1d048c4beb664022716590f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44299 } }
I20260620 20:12:54.118892 25964 leader_election.cc:302] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78bf931ba1d048c4beb664022716590f; no voters:
I20260620 20:12:54.119364 25964 leader_election.cc:288] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:12:54.119540 25969 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:12:54.119812 25969 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f [term 1 LEADER]: Becoming Leader. State: Replica: 78bf931ba1d048c4beb664022716590f, State: Running, Role: LEADER
I20260620 20:12:54.120278 25969 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78bf931ba1d048c4beb664022716590f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44299 } }
I20260620 20:12:54.120890 25964 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:12:54.121855 25971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78bf931ba1d048c4beb664022716590f. Latest consensus state: current_term: 1 leader_uuid: "78bf931ba1d048c4beb664022716590f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78bf931ba1d048c4beb664022716590f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44299 } } }
I20260620 20:12:54.121994 25971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:54.122499 25970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78bf931ba1d048c4beb664022716590f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78bf931ba1d048c4beb664022716590f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44299 } } }
I20260620 20:12:54.122848 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:12:54.123706 25970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f [sys.catalog]: This master's current role is: LEADER
W20260620 20:12:54.126188 25984 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 20:12:54.126261 25984 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 20:12:54.126447 25978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:12:54.127202 25978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:12:54.132368 25978 catalog_manager.cc:1347] Generated new cluster ID: 68593ce9683b424e9d33efe6bbae9acf
I20260620 20:12:54.132437 25978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:12:54.135052 25967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:54.125068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59403 (local address 127.0.0.1:44299)
0620 20:12:54.125177 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:54.125180 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:54.130486 (+ 5306us) server_negotiation.cc:407] Connection header received
0620 20:12:54.130637 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:54.130640 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:54.130690 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:54.130783 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:54.131079 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:54.131851 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:54.133176 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:54.133544 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:54.133647 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:54.134593 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:54.134597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:54.134600 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:12:54.134612 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:54.134650 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:12:54.134662 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:54.134738 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:54.134878 (+ 140us) server_negotiation.cc:299] Negotiation successful
0620 20:12:54.134917 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 20:12:54.142589 25978 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:12:54.143946 25978 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:12:54.149821 25978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78bf931ba1d048c4beb664022716590f: Generated new TSK 0
I20260620 20:12:54.150539 25978 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:12:54.158584 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 25896
W20260620 20:12:54.165099 25881 connection.cc:535] client connection to 127.0.0.1:44299 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44299): Connection reset by peer (error 104)
2026-06-20T20:12:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1362 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:12:54 dist-test-slave-l8h4 krb5kdc[25995](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:12:54 dist-test-slave-l8h4 krb5kdc[25995](info): set up 2 sockets
Jun 20 20:12:54 dist-test-slave-l8h4 krb5kdc[25995](info): commencing operation
krb5kdc: starting...
W20260620 20:12:55.135293 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.956s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:12:55 dist-test-slave-l8h4 krb5kdc[25995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:12:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:12:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 20:12:55.261767 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45581
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45581
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:12:55.372967 26008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:55.373292 26008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:55.373353 26008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:55.377552 26008 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:12:55.377619 26008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:55.377646 26008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:55.377667 26008 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:12:55.377687 26008 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:12:55.383319 26008 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45581
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:55.384379 26008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:55.385362 26008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 20:12:55.394011 26008 server_base.cc:1034] running on GCE node
W20260620 20:12:55.393918 26013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:55.393918 26014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:55.393959 26016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:55.394742 26008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:55.395794 26008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:55.396963 26008 hybrid_clock.cc:648] HybridClock initialized: now 1781986375396938 us; error 37 us; skew 500 ppm
Jun 20 20:12:55 dist-test-slave-l8h4 krb5kdc[25995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:55.399554 26008 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 20:12:55.400380 26008 webserver.cc:460] Webserver started at http://127.0.0.1:40181/ using document root <none> and password file <none>
I20260620 20:12:55.400908 26008 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:55.400972 26008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:55.401147 26008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:55.403313 26008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "83abe0deae0348119b731b7a5c0f6f56"
format_stamp: "Formatted at 2026-06-20 20:12:55 on dist-test-slave-l8h4"
server_key: "1f2ed9ccf452ebc19e1e86721e1ab2e6"
server_key_iv: "a9e0fecece487aaa7d4f48bb7cc0a084"
server_key_version: "encryptionkey@0"
I20260620 20:12:55.403847 26008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "83abe0deae0348119b731b7a5c0f6f56"
format_stamp: "Formatted at 2026-06-20 20:12:55 on dist-test-slave-l8h4"
server_key: "1f2ed9ccf452ebc19e1e86721e1ab2e6"
server_key_iv: "a9e0fecece487aaa7d4f48bb7cc0a084"
server_key_version: "encryptionkey@0"
I20260620 20:12:55.407352 26008 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 20:12:55.409756 26023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:55.411178 26008 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:12:55.411298 26008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "83abe0deae0348119b731b7a5c0f6f56"
format_stamp: "Formatted at 2026-06-20 20:12:55 on dist-test-slave-l8h4"
server_key: "1f2ed9ccf452ebc19e1e86721e1ab2e6"
server_key_iv: "a9e0fecece487aaa7d4f48bb7cc0a084"
server_key_version: "encryptionkey@0"
I20260620 20:12:55.411401 26008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:55.415972 26008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:55.418915 26008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:55.419139 26008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:55.426122 26008 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45581
I20260620 20:12:55.426218 26075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45581 every 8 connection(s)
I20260620 20:12:55.427062 26008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:12:55.428156 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 26008
I20260620 20:12:55.428299 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:12:55.428606 5486 external_mini_cluster.cc:1428] Setting key 3504f3e6de78c1ebb434ac58343098cc
I20260620 20:12:55.430491 26076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:55.434233 26079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:55.430057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57091 (local address 127.0.0.1:45581)
0620 20:12:55.430539 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:55.430566 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:55.430609 (+ 43us) server_negotiation.cc:407] Connection header received
0620 20:12:55.431276 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:55.431293 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:55.431580 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:55.431875 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:55.432931 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:55.432944 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:55.432956 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:12:55.432978 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:55.433044 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:12:55.433070 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:55.433237 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:55.433363 (+ 126us) server_negotiation.cc:299] Negotiation successful
0620 20:12:55.433517 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":132,"threads_started":1}
I20260620 20:12:55.436733 26076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56: Bootstrap starting.
I20260620 20:12:55.437059 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:12:55.438928 26076 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:55.439590 26076 log.cc:824] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:55.440529 26079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:55.438981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42659 (local address 127.0.0.1:45581)
0620 20:12:55.439081 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:55.439084 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:55.439340 (+ 256us) server_negotiation.cc:407] Connection header received
0620 20:12:55.439437 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:55.439440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:55.439471 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:55.439530 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:55.440183 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:55.440187 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:55.440189 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:12:55.440201 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:55.440226 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:12:55.440236 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:55.440304 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:55.440379 (+ 75us) server_negotiation.cc:299] Negotiation successful
0620 20:12:55.440414 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 20:12:55.441686 26076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56: No bootstrap required, opened a new log
I20260620 20:12:55.445041 26076 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83abe0deae0348119b731b7a5c0f6f56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45581 } }
I20260620 20:12:55.445317 26076 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:55.445386 26076 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83abe0deae0348119b731b7a5c0f6f56, State: Initialized, Role: FOLLOWER
I20260620 20:12:55.445794 26076 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83abe0deae0348119b731b7a5c0f6f56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45581 } }
I20260620 20:12:55.445921 26076 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:12:55.445986 26076 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:12:55.446079 26076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:55.446974 26076 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83abe0deae0348119b731b7a5c0f6f56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45581 } }
I20260620 20:12:55.447309 26076 leader_election.cc:302] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83abe0deae0348119b731b7a5c0f6f56; no voters:
I20260620 20:12:55.447583 26076 leader_election.cc:288] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:12:55.447700 26086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:12:55.447921 26086 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56 [term 1 LEADER]: Becoming Leader. State: Replica: 83abe0deae0348119b731b7a5c0f6f56, State: Running, Role: LEADER
I20260620 20:12:55.448221 26086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83abe0deae0348119b731b7a5c0f6f56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45581 } }
I20260620 20:12:55.448730 26076 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:12:55.449725 26087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83abe0deae0348119b731b7a5c0f6f56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83abe0deae0348119b731b7a5c0f6f56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45581 } } }
I20260620 20:12:55.449846 26087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56 [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:55.450085 26088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83abe0deae0348119b731b7a5c0f6f56. Latest consensus state: current_term: 1 leader_uuid: "83abe0deae0348119b731b7a5c0f6f56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83abe0deae0348119b731b7a5c0f6f56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45581 } } }
I20260620 20:12:55.450193 26088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56 [sys.catalog]: This master's current role is: LEADER
W20260620 20:12:55.453532 26101 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 20:12:55.453598 26101 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 20:12:55.453675 26095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:12:55.454569 26095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:12:55.459870 26095 catalog_manager.cc:1347] Generated new cluster ID: 0fd0be3924cd4f0da365f6d8d955d6d9
I20260620 20:12:55.459946 26095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:12:55.485199 26095 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:12:55.485566 26095 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:12:55.491986 26095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83abe0deae0348119b731b7a5c0f6f56: Generated new TSK 0
I20260620 20:12:55.493117 26095 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:12:55.498615 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 26008
2026-06-20T20:12:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1338 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:12:55 dist-test-slave-l8h4 krb5kdc[26107](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:12:55 dist-test-slave-l8h4 krb5kdc[26107](info): set up 2 sockets
Jun 20 20:12:55 dist-test-slave-l8h4 krb5kdc[26107](info): commencing operation
krb5kdc: starting...
W20260620 20:12:56.491369 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.973s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:12:56 dist-test-slave-l8h4 krb5kdc[26107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:12:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:12:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 20:12:56.626646 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43771
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:12:56.750267 26120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:56.750581 26120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:56.750629 26120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:56.755437 26120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:12:56.755518 26120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:56.755544 26120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:56.755560 26120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:12:56.755575 26120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:12:56.761291 26120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:56.763185 26120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:56.764650 26120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:56.775287 26128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:56.775316 26126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:56.775295 26125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:56.775674 26120 server_base.cc:1034] running on GCE node
I20260620 20:12:56.776536 26120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:56.777977 26120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:56.779227 26120 hybrid_clock.cc:648] HybridClock initialized: now 1781986376779210 us; error 39 us; skew 500 ppm
Jun 20 20:12:56 dist-test-slave-l8h4 krb5kdc[26107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986376, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:56.788084 26120 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 20:12:56.789152 26120 webserver.cc:460] Webserver started at http://127.0.0.1:40425/ using document root <none> and password file <none>
I20260620 20:12:56.789829 26120 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:56.789927 26120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:56.790194 26120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:56.792277 26120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "61b3269b954446ffbe82dca7d927df01"
format_stamp: "Formatted at 2026-06-20 20:12:56 on dist-test-slave-l8h4"
server_key: "09c66c1bb073808e13b2bdf827a17b46"
server_key_iv: "2456c93c9acfb79bf3cfc2824f787beb"
server_key_version: "encryptionkey@0"
I20260620 20:12:56.792811 26120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "61b3269b954446ffbe82dca7d927df01"
format_stamp: "Formatted at 2026-06-20 20:12:56 on dist-test-slave-l8h4"
server_key: "09c66c1bb073808e13b2bdf827a17b46"
server_key_iv: "2456c93c9acfb79bf3cfc2824f787beb"
server_key_version: "encryptionkey@0"
I20260620 20:12:56.796881 26120 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260620 20:12:56.799535 26135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:56.801214 26120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 20:12:56.801354 26120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "61b3269b954446ffbe82dca7d927df01"
format_stamp: "Formatted at 2026-06-20 20:12:56 on dist-test-slave-l8h4"
server_key: "09c66c1bb073808e13b2bdf827a17b46"
server_key_iv: "2456c93c9acfb79bf3cfc2824f787beb"
server_key_version: "encryptionkey@0"
I20260620 20:12:56.801462 26120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:56.814236 26120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:56.817773 26120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:56.817953 26120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:56.825189 26120 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43771
I20260620 20:12:56.825253 26187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43771 every 8 connection(s)
I20260620 20:12:56.826186 26120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:12:56.829339 26188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:56.834856 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 26120
I20260620 20:12:56.834982 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:12:56.835047 26188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61b3269b954446ffbe82dca7d927df01: Bootstrap starting.
I20260620 20:12:56.835251 5486 external_mini_cluster.cc:1428] Setting key 23ec46319a59aaa4399897d20d8b516c
I20260620 20:12:56.837829 26188 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61b3269b954446ffbe82dca7d927df01: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:56.838627 26188 log.cc:824] T 00000000000000000000000000000000 P 61b3269b954446ffbe82dca7d927df01: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:56.841523 26188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61b3269b954446ffbe82dca7d927df01: No bootstrap required, opened a new log
Jun 20 20:12:56 dist-test-slave-l8h4 krb5kdc[26107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 20:12:56.846589 26188 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61b3269b954446ffbe82dca7d927df01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b3269b954446ffbe82dca7d927df01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43771 } }
I20260620 20:12:56.846861 26188 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61b3269b954446ffbe82dca7d927df01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:56.846926 26188 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61b3269b954446ffbe82dca7d927df01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b3269b954446ffbe82dca7d927df01, State: Initialized, Role: FOLLOWER
I20260620 20:12:56.847644 26188 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61b3269b954446ffbe82dca7d927df01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b3269b954446ffbe82dca7d927df01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43771 } }
I20260620 20:12:56.847800 26188 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61b3269b954446ffbe82dca7d927df01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:12:56.847872 26188 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61b3269b954446ffbe82dca7d927df01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:12:56.847981 26188 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61b3269b954446ffbe82dca7d927df01 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:56.848918 26188 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61b3269b954446ffbe82dca7d927df01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b3269b954446ffbe82dca7d927df01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43771 } }
I20260620 20:12:56.849359 26188 leader_election.cc:302] T 00000000000000000000000000000000 P 61b3269b954446ffbe82dca7d927df01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61b3269b954446ffbe82dca7d927df01; no voters:
I20260620 20:12:56.849684 26188 leader_election.cc:288] T 00000000000000000000000000000000 P 61b3269b954446ffbe82dca7d927df01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:12:56.849872 26193 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61b3269b954446ffbe82dca7d927df01 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:12:56.850437 26193 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61b3269b954446ffbe82dca7d927df01 [term 1 LEADER]: Becoming Leader. State: Replica: 61b3269b954446ffbe82dca7d927df01, State: Running, Role: LEADER
I20260620 20:12:56.850811 26193 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61b3269b954446ffbe82dca7d927df01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b3269b954446ffbe82dca7d927df01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43771 } }
I20260620 20:12:56.851043 26188 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61b3269b954446ffbe82dca7d927df01 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:12:56.852456 26195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61b3269b954446ffbe82dca7d927df01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61b3269b954446ffbe82dca7d927df01. Latest consensus state: current_term: 1 leader_uuid: "61b3269b954446ffbe82dca7d927df01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b3269b954446ffbe82dca7d927df01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43771 } } }
I20260620 20:12:56.852579 26195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61b3269b954446ffbe82dca7d927df01 [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:56.852684 26191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:56.836914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34953 (local address 127.0.0.1:43771)
0620 20:12:56.837424 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:56.837439 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:56.837481 (+ 42us) server_negotiation.cc:407] Connection header received
0620 20:12:56.838247 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:56.838266 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:56.838620 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:56.838951 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:56.839995 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:56.841065 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:56.842460 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:56.842978 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:56.843159 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:56.846509 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:56.846515 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:56.846530 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:56.846560 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:56.848789 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:56.849379 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:56.849388 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:56.849395 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:56.849771 (+ 376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:56.850309 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:56.850315 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:56.850317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:56.850880 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:56.851137 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:56.851410 (+ 273us) server_negotiation.cc:299] Negotiation successful
0620 20:12:56.851701 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":153,"threads_started":1}
I20260620 20:12:56.853237 26194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61b3269b954446ffbe82dca7d927df01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61b3269b954446ffbe82dca7d927df01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b3269b954446ffbe82dca7d927df01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43771 } } }
I20260620 20:12:56.853430 26194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61b3269b954446ffbe82dca7d927df01 [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:56.853536 26206 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:12:56.856411 26206 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:12:56.857082 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:12:56.862401 26206 catalog_manager.cc:1347] Generated new cluster ID: e3bb0f972952480186eaa02dfff63b45
I20260620 20:12:56.862469 26206 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:12:56.868284 26191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:56.859565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45345 (local address 127.0.0.1:43771)
0620 20:12:56.859690 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:56.859694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:56.859985 (+ 291us) server_negotiation.cc:407] Connection header received
0620 20:12:56.860103 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:56.860106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:56.860167 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:56.860251 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:56.861089 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:56.861816 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:56.863084 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:56.863422 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:56.863525 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:56.865101 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:56.865104 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:56.865107 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:12:56.865131 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:56.866646 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:56.867071 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:56.867074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:56.867076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:56.867147 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:12:56.867380 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:12:56.867383 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:12:56.867385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:12:56.867576 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:56.867683 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:56.867891 (+ 208us) server_negotiation.cc:299] Negotiation successful
0620 20:12:56.867999 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 20:12:56.878396 26206 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:12:56.879639 26206 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:12:56.890645 26206 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61b3269b954446ffbe82dca7d927df01: Generated new TSK 0
I20260620 20:12:56.891534 26206 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:12:56.897073 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 26120
2026-06-20T20:12:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1398 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:12:56 dist-test-slave-l8h4 krb5kdc[26219](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:12:56 dist-test-slave-l8h4 krb5kdc[26219](info): set up 2 sockets
Jun 20 20:12:56 dist-test-slave-l8h4 krb5kdc[26219](info): commencing operation
krb5kdc: starting...
W20260620 20:12:57.907624 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.991s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:12:58 dist-test-slave-l8h4 krb5kdc[26219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:12:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:12:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 20:12:58.055809 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40477
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:12:58.181310 26232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:58.181591 26232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:58.181632 26232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:58.185779 26232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:12:58.185856 26232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:58.185875 26232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:58.185891 26232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:12:58.185904 26232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:12:58.191540 26232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:58.192721 26232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:58.193771 26232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:58.202728 26240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:58.202683 26238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:58.202701 26237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:58.203078 26232 server_base.cc:1034] running on GCE node
I20260620 20:12:58.203780 26232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:58.204947 26232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:58.206198 26232 hybrid_clock.cc:648] HybridClock initialized: now 1781986378206170 us; error 39 us; skew 500 ppm
Jun 20 20:12:58 dist-test-slave-l8h4 krb5kdc[26219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986378, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:58.209395 26232 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 20:12:58.210932 26232 webserver.cc:460] Webserver started at http://127.0.0.1:43501/ using document root <none> and password file <none>
I20260620 20:12:58.211534 26232 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:58.211712 26232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:58.211968 26232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:58.214241 26232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "59015115b04742c096e3dfbb5bb7328b"
format_stamp: "Formatted at 2026-06-20 20:12:58 on dist-test-slave-l8h4"
server_key: "b0d541afb8dcc2e63f97fecf29ea6866"
server_key_iv: "f7d54bbb7009b9df0c36bf54df07679c"
server_key_version: "encryptionkey@0"
I20260620 20:12:58.214797 26232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "59015115b04742c096e3dfbb5bb7328b"
format_stamp: "Formatted at 2026-06-20 20:12:58 on dist-test-slave-l8h4"
server_key: "b0d541afb8dcc2e63f97fecf29ea6866"
server_key_iv: "f7d54bbb7009b9df0c36bf54df07679c"
server_key_version: "encryptionkey@0"
I20260620 20:12:58.219162 26232 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 20:12:58.222162 26247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:58.223559 26232 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:12:58.223727 26232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "59015115b04742c096e3dfbb5bb7328b"
format_stamp: "Formatted at 2026-06-20 20:12:58 on dist-test-slave-l8h4"
server_key: "b0d541afb8dcc2e63f97fecf29ea6866"
server_key_iv: "f7d54bbb7009b9df0c36bf54df07679c"
server_key_version: "encryptionkey@0"
I20260620 20:12:58.223840 26232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:58.241569 26232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:58.244921 26232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:58.245117 26232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:58.252480 26232 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40477
I20260620 20:12:58.252712 26299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40477 every 8 connection(s)
I20260620 20:12:58.253471 26232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:12:58.253813 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 26232
I20260620 20:12:58.253952 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:12:58.254489 5486 external_mini_cluster.cc:1428] Setting key 9aff6b8592f6e8cc15bdd4e503c0424c
I20260620 20:12:58.256906 26300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:58.262893 26300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59015115b04742c096e3dfbb5bb7328b: Bootstrap starting.
I20260620 20:12:58.264034 26303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:58.255966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43983 (local address 127.0.0.1:40477)
0620 20:12:58.256432 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:58.256446 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:58.256481 (+ 35us) server_negotiation.cc:407] Connection header received
0620 20:12:58.257212 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:58.257230 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:58.257572 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:58.257900 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:58.258323 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:58.259299 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:58.260156 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:58.260638 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:58.260780 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:58.262045 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:58.262053 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:58.262073 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:12:58.262679 (+ 606us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:58.262740 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:12:58.262767 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:58.262869 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:58.263071 (+ 202us) server_negotiation.cc:299] Negotiation successful
0620 20:12:58.263274 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":146,"threads_started":1}
I20260620 20:12:58.266412 26300 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59015115b04742c096e3dfbb5bb7328b: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:58.267663 26300 log.cc:824] T 00000000000000000000000000000000 P 59015115b04742c096e3dfbb5bb7328b: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:58.267814 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:12:58.270184 26300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59015115b04742c096e3dfbb5bb7328b: No bootstrap required, opened a new log
I20260620 20:12:58.274595 26300 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59015115b04742c096e3dfbb5bb7328b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59015115b04742c096e3dfbb5bb7328b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40477 } }
I20260620 20:12:58.274910 26300 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59015115b04742c096e3dfbb5bb7328b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:58.274973 26300 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59015115b04742c096e3dfbb5bb7328b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59015115b04742c096e3dfbb5bb7328b, State: Initialized, Role: FOLLOWER
I20260620 20:12:58.274991 26303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:58.269988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44813 (local address 127.0.0.1:40477)
0620 20:12:58.270116 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:58.270119 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:58.270263 (+ 144us) server_negotiation.cc:407] Connection header received
0620 20:12:58.270383 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:58.270386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:58.270426 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:58.270490 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:58.270959 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:58.271528 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:58.272677 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:12:58.272917 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:12:58.273006 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:12:58.274247 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:58.274251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:58.274254 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:12:58.274269 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:58.274302 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:12:58.274315 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:58.274382 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:58.274609 (+ 227us) server_negotiation.cc:299] Negotiation successful
0620 20:12:58.274784 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 20:12:58.275462 26300 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59015115b04742c096e3dfbb5bb7328b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59015115b04742c096e3dfbb5bb7328b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40477 } }
I20260620 20:12:58.275923 26300 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59015115b04742c096e3dfbb5bb7328b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:12:58.276006 26300 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59015115b04742c096e3dfbb5bb7328b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:12:58.276108 26300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59015115b04742c096e3dfbb5bb7328b [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:58.277237 26300 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59015115b04742c096e3dfbb5bb7328b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59015115b04742c096e3dfbb5bb7328b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40477 } }
I20260620 20:12:58.277565 26300 leader_election.cc:302] T 00000000000000000000000000000000 P 59015115b04742c096e3dfbb5bb7328b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59015115b04742c096e3dfbb5bb7328b; no voters:
I20260620 20:12:58.278121 26300 leader_election.cc:288] T 00000000000000000000000000000000 P 59015115b04742c096e3dfbb5bb7328b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:12:58.278290 26310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59015115b04742c096e3dfbb5bb7328b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:12:58.278800 26310 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59015115b04742c096e3dfbb5bb7328b [term 1 LEADER]: Becoming Leader. State: Replica: 59015115b04742c096e3dfbb5bb7328b, State: Running, Role: LEADER
I20260620 20:12:58.279146 26310 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59015115b04742c096e3dfbb5bb7328b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59015115b04742c096e3dfbb5bb7328b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40477 } }
I20260620 20:12:58.279866 26300 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59015115b04742c096e3dfbb5bb7328b [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:12:58.280928 26312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59015115b04742c096e3dfbb5bb7328b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59015115b04742c096e3dfbb5bb7328b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59015115b04742c096e3dfbb5bb7328b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40477 } } }
I20260620 20:12:58.281054 26312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59015115b04742c096e3dfbb5bb7328b [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:58.281006 26311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59015115b04742c096e3dfbb5bb7328b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59015115b04742c096e3dfbb5bb7328b. Latest consensus state: current_term: 1 leader_uuid: "59015115b04742c096e3dfbb5bb7328b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59015115b04742c096e3dfbb5bb7328b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40477 } } }
I20260620 20:12:58.281812 26311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59015115b04742c096e3dfbb5bb7328b [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:58.282102 26317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:12:58.285079 26317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:12:58.290371 26317 catalog_manager.cc:1347] Generated new cluster ID: 5c9a39d579e2404981b75f8f5b98d79e
I20260620 20:12:58.290439 26317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:12:58.318710 26317 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:12:58.320168 26317 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:12:58.328553 26317 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59015115b04742c096e3dfbb5bb7328b: Generated new TSK 0
I20260620 20:12:58.329978 26317 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:12:58.347410 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 26232
2026-06-20T20:12:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1450 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:12:58 dist-test-slave-l8h4 krb5kdc[26331](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:12:58 dist-test-slave-l8h4 krb5kdc[26331](info): set up 2 sockets
Jun 20 20:12:58 dist-test-slave-l8h4 krb5kdc[26331](info): commencing operation
krb5kdc: starting...
W20260620 20:12:59.343307 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.977s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:12:59 dist-test-slave-l8h4 krb5kdc[26331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:12:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:12:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 20:12:59.464841 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38809
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:12:59.574394 26344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:12:59.574656 26344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:12:59.574715 26344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:12:59.578814 26344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:12:59.578877 26344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:12:59.578902 26344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:12:59.578926 26344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:12:59.578954 26344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:12:59.584183 26344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:12:59.585166 26344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:12:59.586126 26344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:12:59.594203 26349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:12:59.594199 26350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:59.594300 26344 server_base.cc:1034] running on GCE node
W20260620 20:12:59.594193 26352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:12:59.595081 26344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:12:59.596110 26344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:12:59.597288 26344 hybrid_clock.cc:648] HybridClock initialized: now 1781986379597260 us; error 39 us; skew 500 ppm
Jun 20 20:12:59 dist-test-slave-l8h4 krb5kdc[26331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986379, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:12:59.600395 26344 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 20:12:59.601639 26344 webserver.cc:460] Webserver started at http://127.0.0.1:33769/ using document root <none> and password file <none>
I20260620 20:12:59.602506 26344 fs_manager.cc:359] Metadata directory not provided
I20260620 20:12:59.602609 26344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:12:59.602823 26344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:12:59.604847 26344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "49c3ad4e32bf4ff582f38d75d90a5cad"
format_stamp: "Formatted at 2026-06-20 20:12:59 on dist-test-slave-l8h4"
server_key: "a48b0e5acdbd039729b75869edb715e4"
server_key_iv: "6fc948b6a275bc3143c6315ec8524176"
server_key_version: "encryptionkey@0"
I20260620 20:12:59.605379 26344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "49c3ad4e32bf4ff582f38d75d90a5cad"
format_stamp: "Formatted at 2026-06-20 20:12:59 on dist-test-slave-l8h4"
server_key: "a48b0e5acdbd039729b75869edb715e4"
server_key_iv: "6fc948b6a275bc3143c6315ec8524176"
server_key_version: "encryptionkey@0"
I20260620 20:12:59.609858 26344 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260620 20:12:59.612582 26359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:12:59.613920 26344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 20:12:59.614096 26344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "49c3ad4e32bf4ff582f38d75d90a5cad"
format_stamp: "Formatted at 2026-06-20 20:12:59 on dist-test-slave-l8h4"
server_key: "a48b0e5acdbd039729b75869edb715e4"
server_key_iv: "6fc948b6a275bc3143c6315ec8524176"
server_key_version: "encryptionkey@0"
I20260620 20:12:59.614265 26344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:12:59.618793 26344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:12:59.621968 26344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:12:59.622124 26344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:12:59.628436 26344 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38809
I20260620 20:12:59.628593 26411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38809 every 8 connection(s)
I20260620 20:12:59.629335 26344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:12:59.630694 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 26344
I20260620 20:12:59.630831 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:12:59.631145 5486 external_mini_cluster.cc:1428] Setting key 8ea12470e79729bd039d7243c79d3fce
I20260620 20:12:59.632509 26412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:12:59.635898 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:59.632491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47453 (local address 127.0.0.1:38809)
0620 20:12:59.632892 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:59.632907 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:59.632940 (+ 33us) server_negotiation.cc:407] Connection header received
0620 20:12:59.633550 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:59.633564 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:59.633828 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:59.634198 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:59.634851 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:59.634856 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:59.634866 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:12:59.634876 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:59.634924 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:12:59.634943 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:59.635060 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:59.635135 (+ 75us) server_negotiation.cc:299] Negotiation successful
0620 20:12:59.635272 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":121,"threads_started":1}
I20260620 20:12:59.638108 26412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49c3ad4e32bf4ff582f38d75d90a5cad: Bootstrap starting.
I20260620 20:12:59.638437 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:12:59.640403 26412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49c3ad4e32bf4ff582f38d75d90a5cad: Neither blocks nor log segments found. Creating new log.
I20260620 20:12:59.641173 26412 log.cc:824] T 00000000000000000000000000000000 P 49c3ad4e32bf4ff582f38d75d90a5cad: Log is configured to *not* fsync() on all Append() calls
I20260620 20:12:59.641767 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:12:59.640234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56795 (local address 127.0.0.1:38809)
0620 20:12:59.640329 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0620 20:12:59.640333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:12:59.640478 (+ 145us) server_negotiation.cc:407] Connection header received
0620 20:12:59.640580 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:12:59.640583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:12:59.640630 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:12:59.640693 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:12:59.641378 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:12:59.641381 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:12:59.641383 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:12:59.641392 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:12:59.641416 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:12:59.641426 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:12:59.641497 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 20:12:59.641622 (+ 125us) server_negotiation.cc:299] Negotiation successful
0620 20:12:59.641655 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 20:12:59.643184 26412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49c3ad4e32bf4ff582f38d75d90a5cad: No bootstrap required, opened a new log
I20260620 20:12:59.646723 26412 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49c3ad4e32bf4ff582f38d75d90a5cad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c3ad4e32bf4ff582f38d75d90a5cad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38809 } }
I20260620 20:12:59.646938 26412 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49c3ad4e32bf4ff582f38d75d90a5cad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:12:59.646977 26412 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49c3ad4e32bf4ff582f38d75d90a5cad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c3ad4e32bf4ff582f38d75d90a5cad, State: Initialized, Role: FOLLOWER
I20260620 20:12:59.647337 26412 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49c3ad4e32bf4ff582f38d75d90a5cad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c3ad4e32bf4ff582f38d75d90a5cad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38809 } }
I20260620 20:12:59.647454 26412 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49c3ad4e32bf4ff582f38d75d90a5cad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:12:59.647508 26412 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49c3ad4e32bf4ff582f38d75d90a5cad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:12:59.647588 26412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49c3ad4e32bf4ff582f38d75d90a5cad [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:12:59.648408 26412 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49c3ad4e32bf4ff582f38d75d90a5cad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c3ad4e32bf4ff582f38d75d90a5cad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38809 } }
I20260620 20:12:59.648696 26412 leader_election.cc:302] T 00000000000000000000000000000000 P 49c3ad4e32bf4ff582f38d75d90a5cad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49c3ad4e32bf4ff582f38d75d90a5cad; no voters:
I20260620 20:12:59.648947 26412 leader_election.cc:288] T 00000000000000000000000000000000 P 49c3ad4e32bf4ff582f38d75d90a5cad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:12:59.649058 26422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49c3ad4e32bf4ff582f38d75d90a5cad [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:12:59.649266 26422 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49c3ad4e32bf4ff582f38d75d90a5cad [term 1 LEADER]: Becoming Leader. State: Replica: 49c3ad4e32bf4ff582f38d75d90a5cad, State: Running, Role: LEADER
I20260620 20:12:59.649570 26422 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49c3ad4e32bf4ff582f38d75d90a5cad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c3ad4e32bf4ff582f38d75d90a5cad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38809 } }
I20260620 20:12:59.650018 26412 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49c3ad4e32bf4ff582f38d75d90a5cad [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:12:59.651131 26423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49c3ad4e32bf4ff582f38d75d90a5cad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49c3ad4e32bf4ff582f38d75d90a5cad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c3ad4e32bf4ff582f38d75d90a5cad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38809 } } }
I20260620 20:12:59.651254 26423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49c3ad4e32bf4ff582f38d75d90a5cad [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:59.651491 26424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49c3ad4e32bf4ff582f38d75d90a5cad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49c3ad4e32bf4ff582f38d75d90a5cad. Latest consensus state: current_term: 1 leader_uuid: "49c3ad4e32bf4ff582f38d75d90a5cad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c3ad4e32bf4ff582f38d75d90a5cad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38809 } } }
I20260620 20:12:59.651578 26424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49c3ad4e32bf4ff582f38d75d90a5cad [sys.catalog]: This master's current role is: LEADER
I20260620 20:12:59.651801 26431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:12:59.654913 26431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:12:59.660250 26431 catalog_manager.cc:1347] Generated new cluster ID: dbf04be89b9d4403b9f292fa2c2c07ae
I20260620 20:12:59.660317 26431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:12:59.671912 26431 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:12:59.672202 26431 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:12:59.678810 26431 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49c3ad4e32bf4ff582f38d75d90a5cad: Generated new TSK 0
I20260620 20:12:59.679545 26431 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:12:59.689096 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 26344
2026-06-20T20:12:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1341 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:12:59 dist-test-slave-l8h4 krb5kdc[26443](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:12:59 dist-test-slave-l8h4 krb5kdc[26443](info): set up 2 sockets
Jun 20 20:12:59 dist-test-slave-l8h4 krb5kdc[26443](info): commencing operation
krb5kdc: starting...
W20260620 20:13:00.662890 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.955s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:13:00 dist-test-slave-l8h4 krb5kdc[26443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:13:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:13:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 20:13:00.779141 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36659
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:13:00.891479 26456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:13:00.891748 26456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:13:00.891793 26456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:13:00.896112 26456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:13:00.896183 26456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:13:00.896203 26456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:13:00.896219 26456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:13:00.896232 26456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:13:00.901623 26456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:13:00.902760 26456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:13:00.903842 26456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:13:00.913375 26462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:00.913378 26464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:00.913611 26461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:00.913770 26456 server_base.cc:1034] running on GCE node
I20260620 20:13:00.914399 26456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:13:00.915525 26456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:13:00.916723 26456 hybrid_clock.cc:648] HybridClock initialized: now 1781986380916697 us; error 36 us; skew 500 ppm
Jun 20 20:13:00 dist-test-slave-l8h4 krb5kdc[26443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986380, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:13:00.920503 26456 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 20:13:00.921741 26456 webserver.cc:460] Webserver started at http://127.0.0.1:33141/ using document root <none> and password file <none>
I20260620 20:13:00.922654 26456 fs_manager.cc:359] Metadata directory not provided
I20260620 20:13:00.922756 26456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:13:00.922966 26456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:13:00.925179 26456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "de47e7f930414b2fa5e99ea60420cc09"
format_stamp: "Formatted at 2026-06-20 20:13:00 on dist-test-slave-l8h4"
server_key: "41c0b42fa08d6245d830402d175b7b3f"
server_key_iv: "189b1e9e3522b494d6fed93c8e9a8a9e"
server_key_version: "encryptionkey@0"
I20260620 20:13:00.925751 26456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "de47e7f930414b2fa5e99ea60420cc09"
format_stamp: "Formatted at 2026-06-20 20:13:00 on dist-test-slave-l8h4"
server_key: "41c0b42fa08d6245d830402d175b7b3f"
server_key_iv: "189b1e9e3522b494d6fed93c8e9a8a9e"
server_key_version: "encryptionkey@0"
I20260620 20:13:00.929651 26456 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 20:13:00.932161 26471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:13:00.933951 26456 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260620 20:13:00.934207 26456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "de47e7f930414b2fa5e99ea60420cc09"
format_stamp: "Formatted at 2026-06-20 20:13:00 on dist-test-slave-l8h4"
server_key: "41c0b42fa08d6245d830402d175b7b3f"
server_key_iv: "189b1e9e3522b494d6fed93c8e9a8a9e"
server_key_version: "encryptionkey@0"
I20260620 20:13:00.934374 26456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:13:00.956725 26456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:13:00.960443 26456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:13:00.960651 26456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:13:00.968310 26456 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36659
I20260620 20:13:00.968461 26523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36659 every 8 connection(s)
I20260620 20:13:00.969375 26456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:13:00.972294 26524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:13:00.976127 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 26456
I20260620 20:13:00.976275 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:13:00.976567 5486 external_mini_cluster.cc:1428] Setting key 6bea9e058aa7486ff21a6a073d715115
I20260620 20:13:00.978487 26524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09: Bootstrap starting.
I20260620 20:13:00.981362 26524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09: Neither blocks nor log segments found. Creating new log.
I20260620 20:13:00.982262 26524 log.cc:824] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09: Log is configured to *not* fsync() on all Append() calls
I20260620 20:13:00.985054 26524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09: No bootstrap required, opened a new log
Jun 20 20:13:00 dist-test-slave-l8h4 krb5kdc[26443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 20:13:00.989778 26524 raft_consensus.cc:348] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de47e7f930414b2fa5e99ea60420cc09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36659 } }
I20260620 20:13:00.990084 26524 raft_consensus.cc:374] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:13:00.990155 26524 raft_consensus.cc:729] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de47e7f930414b2fa5e99ea60420cc09, State: Initialized, Role: FOLLOWER
I20260620 20:13:00.990646 26524 consensus_queue.cc:260] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de47e7f930414b2fa5e99ea60420cc09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36659 } }
I20260620 20:13:00.990779 26524 raft_consensus.cc:388] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:13:00.990836 26524 raft_consensus.cc:482] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:13:00.990932 26524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:13:00.991812 26524 raft_consensus.cc:504] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de47e7f930414b2fa5e99ea60420cc09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36659 } }
I20260620 20:13:00.992137 26524 leader_election.cc:302] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de47e7f930414b2fa5e99ea60420cc09; no voters:
I20260620 20:13:00.992436 26524 leader_election.cc:288] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:13:00.992576 26529 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:13:00.992798 26529 raft_consensus.cc:686] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09 [term 1 LEADER]: Becoming Leader. State: Replica: de47e7f930414b2fa5e99ea60420cc09, State: Running, Role: LEADER
I20260620 20:13:00.993108 26529 consensus_queue.cc:237] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de47e7f930414b2fa5e99ea60420cc09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36659 } }
I20260620 20:13:00.993552 26527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:00.977998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:34555 (local address 127.0.0.1:36659)
0620 20:13:00.978506 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:00.978589 (+ 83us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:00.978649 (+ 60us) server_negotiation.cc:407] Connection header received
0620 20:13:00.979491 (+ 842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:00.979513 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:00.979960 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:00.980315 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:00.981012 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:00.982498 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:00.983611 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:00.984461 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:00.984678 (+ 217us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:00.988148 (+ 3470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:00.988156 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:00.988183 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:00.988222 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:00.990372 (+ 2150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:00.990954 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:00.990959 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:00.990963 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:00.991116 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:00.991441 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:00.991445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:00.991446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:00.991849 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:00.992034 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:00.992381 (+ 347us) server_negotiation.cc:299] Negotiation successful
0620 20:13:00.992658 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":121,"threads_started":1}
I20260620 20:13:00.993728 26524 sys_catalog.cc:564] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:13:00.994807 26530 sys_catalog.cc:455] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de47e7f930414b2fa5e99ea60420cc09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de47e7f930414b2fa5e99ea60420cc09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36659 } } }
I20260620 20:13:00.994935 26530 sys_catalog.cc:458] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09 [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:00.995203 26531 sys_catalog.cc:455] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de47e7f930414b2fa5e99ea60420cc09. Latest consensus state: current_term: 1 leader_uuid: "de47e7f930414b2fa5e99ea60420cc09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de47e7f930414b2fa5e99ea60420cc09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36659 } } }
I20260620 20:13:00.995445 26531 sys_catalog.cc:458] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09 [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:00.996969 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260620 20:13:00.998821 26544 catalog_manager.cc:1558] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 20:13:00.998904 26544 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 20:13:00.999001 26539 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:13:00.999825 26539 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:13:01.006280 26539 catalog_manager.cc:1347] Generated new cluster ID: b9afe4fbe659426ab1ebff460d61a5e3
I20260620 20:13:01.006353 26539 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:13:01.012351 26527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:00.999367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:51875 (local address 127.0.0.1:36659)
0620 20:13:00.999486 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:00.999491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:01.001579 (+ 2088us) server_negotiation.cc:407] Connection header received
0620 20:13:01.001777 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:01.001780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:01.001829 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:01.001878 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:01.002881 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:01.003682 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:01.004631 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:01.004959 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:01.005064 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:01.006859 (+ 1795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:01.006863 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:01.006866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:01.006892 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:01.009291 (+ 2399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:01.010423 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:01.010430 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:01.010432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:01.010591 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:01.011263 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:01.011267 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:01.011269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:01.011524 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:01.011674 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:01.012064 (+ 390us) server_negotiation.cc:299] Negotiation successful
0620 20:13:01.012177 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 20:13:01.018011 26539 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:13:01.019490 26539 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:13:01.034528 26539 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de47e7f930414b2fa5e99ea60420cc09: Generated new TSK 0
I20260620 20:13:01.035322 26539 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:13:01.041535 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 26456
W20260620 20:13:01.046283 26441 connection.cc:535] client connection to 127.0.0.1:36659 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36659): Connection reset by peer (error 104)
2026-06-20T20:13:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1352 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:13:01 dist-test-slave-l8h4 krb5kdc[26555](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:13:01 dist-test-slave-l8h4 krb5kdc[26555](info): set up 2 sockets
Jun 20 20:13:01 dist-test-slave-l8h4 krb5kdc[26555](info): commencing operation
krb5kdc: starting...
W20260620 20:13:02.019968 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.959s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:13:02 dist-test-slave-l8h4 krb5kdc[26555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:13:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:13:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 20:13:02.149134 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40741
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:13:02.257517 26568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:13:02.257818 26568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:13:02.257864 26568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:13:02.262220 26568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:13:02.262305 26568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:13:02.262323 26568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:13:02.262339 26568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:13:02.262353 26568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:13:02.267894 26568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:13:02.268945 26568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:13:02.269933 26568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 20:13:02.278080 26568 server_base.cc:1034] running on GCE node
W20260620 20:13:02.278031 26576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:02.278031 26574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:02.278066 26573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:02.278863 26568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:13:02.279964 26568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:13:02.281132 26568 hybrid_clock.cc:648] HybridClock initialized: now 1781986382281118 us; error 27 us; skew 500 ppm
Jun 20 20:13:02 dist-test-slave-l8h4 krb5kdc[26555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986382, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:13:02.284094 26568 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 20:13:02.285025 26568 webserver.cc:460] Webserver started at http://127.0.0.1:42455/ using document root <none> and password file <none>
I20260620 20:13:02.285538 26568 fs_manager.cc:359] Metadata directory not provided
I20260620 20:13:02.285602 26568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:13:02.285780 26568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:13:02.287803 26568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "d6eccc760c1c4c068cc8c0da4bc0a470"
format_stamp: "Formatted at 2026-06-20 20:13:02 on dist-test-slave-l8h4"
server_key: "3ccc8953b4b4b45afaac4dae5bb793a1"
server_key_iv: "d522c0d7dc5f434171946e2c7a0d4d5b"
server_key_version: "encryptionkey@0"
I20260620 20:13:02.288340 26568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "d6eccc760c1c4c068cc8c0da4bc0a470"
format_stamp: "Formatted at 2026-06-20 20:13:02 on dist-test-slave-l8h4"
server_key: "3ccc8953b4b4b45afaac4dae5bb793a1"
server_key_iv: "d522c0d7dc5f434171946e2c7a0d4d5b"
server_key_version: "encryptionkey@0"
I20260620 20:13:02.291918 26568 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260620 20:13:02.294581 26583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:13:02.295928 26568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:13:02.296099 26568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "d6eccc760c1c4c068cc8c0da4bc0a470"
format_stamp: "Formatted at 2026-06-20 20:13:02 on dist-test-slave-l8h4"
server_key: "3ccc8953b4b4b45afaac4dae5bb793a1"
server_key_iv: "d522c0d7dc5f434171946e2c7a0d4d5b"
server_key_version: "encryptionkey@0"
I20260620 20:13:02.296206 26568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:13:02.317358 26568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:13:02.320429 26568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:13:02.320606 26568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:13:02.327821 26568 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40741
I20260620 20:13:02.327847 26635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40741 every 8 connection(s)
I20260620 20:13:02.328737 26568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:13:02.331594 26636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:13:02.335388 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 26568
I20260620 20:13:02.335530 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:13:02.335814 5486 external_mini_cluster.cc:1428] Setting key 16e6a3799e9e9e70d0866784719db98b
I20260620 20:13:02.337396 26636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470: Bootstrap starting.
I20260620 20:13:02.339576 26636 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470: Neither blocks nor log segments found. Creating new log.
I20260620 20:13:02.340271 26636 log.cc:824] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470: Log is configured to *not* fsync() on all Append() calls
I20260620 20:13:02.342031 26636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470: No bootstrap required, opened a new log
I20260620 20:13:02.344378 26639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:02.337188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:43763 (local address 127.0.0.1:40741)
0620 20:13:02.337638 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:02.337654 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:02.337690 (+ 36us) server_negotiation.cc:407] Connection header received
0620 20:13:02.338375 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:02.338393 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:02.338727 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:02.339020 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:02.339408 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:02.340420 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:02.341275 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:02.341741 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:02.341891 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:02.342795 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:02.342800 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:02.342814 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:13:02.343190 (+ 376us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:02.343241 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:13:02.343264 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:02.343378 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:02.343562 (+ 184us) server_negotiation.cc:299] Negotiation successful
0620 20:13:02.343687 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":118,"threads_started":1}
I20260620 20:13:02.345742 26636 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6eccc760c1c4c068cc8c0da4bc0a470" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40741 } }
I20260620 20:13:02.346014 26636 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:13:02.346089 26636 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6eccc760c1c4c068cc8c0da4bc0a470, State: Initialized, Role: FOLLOWER
I20260620 20:13:02.346561 26636 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6eccc760c1c4c068cc8c0da4bc0a470" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40741 } }
I20260620 20:13:02.346709 26636 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:13:02.346771 26636 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:13:02.346863 26636 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:13:02.347513 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:13:02.347944 26636 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6eccc760c1c4c068cc8c0da4bc0a470" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40741 } }
I20260620 20:13:02.348315 26636 leader_election.cc:302] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6eccc760c1c4c068cc8c0da4bc0a470; no voters:
I20260620 20:13:02.348641 26636 leader_election.cc:288] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:13:02.348737 26641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:13:02.348986 26641 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470 [term 1 LEADER]: Becoming Leader. State: Replica: d6eccc760c1c4c068cc8c0da4bc0a470, State: Running, Role: LEADER
I20260620 20:13:02.349334 26641 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6eccc760c1c4c068cc8c0da4bc0a470" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40741 } }
I20260620 20:13:02.349884 26636 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:13:02.350994 26644 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6eccc760c1c4c068cc8c0da4bc0a470. Latest consensus state: current_term: 1 leader_uuid: "d6eccc760c1c4c068cc8c0da4bc0a470" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6eccc760c1c4c068cc8c0da4bc0a470" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40741 } } }
I20260620 20:13:02.351123 26644 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470 [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:02.351423 26646 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6eccc760c1c4c068cc8c0da4bc0a470" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6eccc760c1c4c068cc8c0da4bc0a470" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40741 } } }
I20260620 20:13:02.351500 26646 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470 [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:02.354275 26639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:02.349997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:35367 (local address 127.0.0.1:40741)
0620 20:13:02.350094 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:02.350098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:02.350111 (+ 13us) server_negotiation.cc:407] Connection header received
0620 20:13:02.350174 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:02.350177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:02.350224 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:02.350302 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:02.350628 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:02.351386 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:02.352501 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:02.352723 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:02.352796 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:02.353716 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:02.353720 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:02.353722 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:13:02.353736 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:02.353763 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:13:02.353775 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:02.353861 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:02.354087 (+ 226us) server_negotiation.cc:299] Negotiation successful
0620 20:13:02.354122 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260620 20:13:02.354888 26661 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 20:13:02.354960 26661 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 20:13:02.355020 26655 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:13:02.355700 26655 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:13:02.362042 26655 catalog_manager.cc:1347] Generated new cluster ID: 422c9cb11698457e8ef4703afad837a9
I20260620 20:13:02.362120 26655 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:13:02.385388 26655 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:13:02.386723 26655 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:13:02.397651 26655 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6eccc760c1c4c068cc8c0da4bc0a470: Generated new TSK 0
I20260620 20:13:02.398334 26655 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:13:02.407196 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 26568
W20260620 20:13:02.412083 26553 connection.cc:535] client connection to 127.0.0.1:40741 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40741): Connection reset by peer (error 104)
2026-06-20T20:13:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1365 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:13:02 dist-test-slave-l8h4 krb5kdc[26667](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:13:02 dist-test-slave-l8h4 krb5kdc[26667](info): set up 2 sockets
Jun 20 20:13:02 dist-test-slave-l8h4 krb5kdc[26667](info): commencing operation
krb5kdc: starting...
W20260620 20:13:03.378975 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.953s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:13:03 dist-test-slave-l8h4 krb5kdc[26667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:13:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:13:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 20:13:03.505950 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33657
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:13:03.615649 26680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:13:03.615945 26680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:13:03.615991 26680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:13:03.620381 26680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:13:03.620445 26680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:13:03.620462 26680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:13:03.620478 26680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:13:03.620492 26680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:13:03.625998 26680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:13:03.627135 26680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:13:03.628123 26680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:13:03.636634 26685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:03.636687 26680 server_base.cc:1034] running on GCE node
W20260620 20:13:03.636646 26688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:03.636660 26686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:03.637560 26680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:13:03.638664 26680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:13:03.639849 26680 hybrid_clock.cc:648] HybridClock initialized: now 1781986383639828 us; error 34 us; skew 500 ppm
Jun 20 20:13:03 dist-test-slave-l8h4 krb5kdc[26667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986383, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:13:03.642735 26680 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 20:13:03.643652 26680 webserver.cc:460] Webserver started at http://127.0.0.1:35687/ using document root <none> and password file <none>
I20260620 20:13:03.644197 26680 fs_manager.cc:359] Metadata directory not provided
I20260620 20:13:03.644265 26680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:13:03.644454 26680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:13:03.646591 26680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "2e15f17435d04461b73f0a0756c245e3"
format_stamp: "Formatted at 2026-06-20 20:13:03 on dist-test-slave-l8h4"
server_key: "6821fc750e6e2d2495a3d2e412b6944f"
server_key_iv: "f445d425704f5499c58fe39b1c706426"
server_key_version: "encryptionkey@0"
I20260620 20:13:03.647131 26680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "2e15f17435d04461b73f0a0756c245e3"
format_stamp: "Formatted at 2026-06-20 20:13:03 on dist-test-slave-l8h4"
server_key: "6821fc750e6e2d2495a3d2e412b6944f"
server_key_iv: "f445d425704f5499c58fe39b1c706426"
server_key_version: "encryptionkey@0"
I20260620 20:13:03.650848 26680 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:13:03.653389 26695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:13:03.654824 26680 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:13:03.654945 26680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "2e15f17435d04461b73f0a0756c245e3"
format_stamp: "Formatted at 2026-06-20 20:13:03 on dist-test-slave-l8h4"
server_key: "6821fc750e6e2d2495a3d2e412b6944f"
server_key_iv: "f445d425704f5499c58fe39b1c706426"
server_key_version: "encryptionkey@0"
I20260620 20:13:03.655077 26680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:13:03.659904 26680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:13:03.663969 26680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:13:03.664170 26680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:13:03.670593 26680 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33657
I20260620 20:13:03.670611 26747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33657 every 8 connection(s)
I20260620 20:13:03.671484 26680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:13:03.672142 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 26680
I20260620 20:13:03.672259 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:13:03.672497 5486 external_mini_cluster.cc:1428] Setting key 420bd65f2444070ebf89f8ce389cbe65
I20260620 20:13:03.674620 26748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:13:03.677837 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:03.673805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:56589 (local address 127.0.0.1:33657)
0620 20:13:03.674223 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:03.674240 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:03.674281 (+ 41us) server_negotiation.cc:407] Connection header received
0620 20:13:03.675093 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:03.675114 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:03.675441 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:03.675786 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:03.676534 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:03.676540 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:03.676555 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:13:03.676566 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:03.676627 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:13:03.676650 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:03.676780 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:03.676848 (+ 68us) server_negotiation.cc:299] Negotiation successful
0620 20:13:03.676991 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":111,"threads_started":1}
I20260620 20:13:03.680549 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:13:03.680933 26748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e15f17435d04461b73f0a0756c245e3: Bootstrap starting.
I20260620 20:13:03.683107 26748 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e15f17435d04461b73f0a0756c245e3: Neither blocks nor log segments found. Creating new log.
I20260620 20:13:03.683769 26748 log.cc:824] T 00000000000000000000000000000000 P 2e15f17435d04461b73f0a0756c245e3: Log is configured to *not* fsync() on all Append() calls
I20260620 20:13:03.683962 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:03.682476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:60339 (local address 127.0.0.1:33657)
0620 20:13:03.682570 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:03.682574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:03.682773 (+ 199us) server_negotiation.cc:407] Connection header received
0620 20:13:03.682873 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:03.682876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:03.682908 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:03.682969 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:03.683601 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:03.683604 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:03.683607 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:13:03.683618 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:03.683651 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:13:03.683666 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:03.683733 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:03.683816 (+ 83us) server_negotiation.cc:299] Negotiation successful
0620 20:13:03.683853 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 20:13:03.685525 26748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e15f17435d04461b73f0a0756c245e3: No bootstrap required, opened a new log
I20260620 20:13:03.689136 26748 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e15f17435d04461b73f0a0756c245e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e15f17435d04461b73f0a0756c245e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33657 } }
I20260620 20:13:03.689348 26748 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e15f17435d04461b73f0a0756c245e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:13:03.689390 26748 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e15f17435d04461b73f0a0756c245e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e15f17435d04461b73f0a0756c245e3, State: Initialized, Role: FOLLOWER
I20260620 20:13:03.689740 26748 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e15f17435d04461b73f0a0756c245e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e15f17435d04461b73f0a0756c245e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33657 } }
I20260620 20:13:03.689847 26748 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e15f17435d04461b73f0a0756c245e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:13:03.689884 26748 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e15f17435d04461b73f0a0756c245e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:13:03.689949 26748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e15f17435d04461b73f0a0756c245e3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:13:03.690796 26748 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e15f17435d04461b73f0a0756c245e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e15f17435d04461b73f0a0756c245e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33657 } }
I20260620 20:13:03.691082 26748 leader_election.cc:302] T 00000000000000000000000000000000 P 2e15f17435d04461b73f0a0756c245e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e15f17435d04461b73f0a0756c245e3; no voters:
I20260620 20:13:03.691349 26748 leader_election.cc:288] T 00000000000000000000000000000000 P 2e15f17435d04461b73f0a0756c245e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:13:03.691488 26758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e15f17435d04461b73f0a0756c245e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:13:03.691704 26758 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e15f17435d04461b73f0a0756c245e3 [term 1 LEADER]: Becoming Leader. State: Replica: 2e15f17435d04461b73f0a0756c245e3, State: Running, Role: LEADER
I20260620 20:13:03.691972 26758 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e15f17435d04461b73f0a0756c245e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e15f17435d04461b73f0a0756c245e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33657 } }
I20260620 20:13:03.692473 26748 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e15f17435d04461b73f0a0756c245e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:13:03.693451 26760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e15f17435d04461b73f0a0756c245e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e15f17435d04461b73f0a0756c245e3. Latest consensus state: current_term: 1 leader_uuid: "2e15f17435d04461b73f0a0756c245e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e15f17435d04461b73f0a0756c245e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33657 } } }
I20260620 20:13:03.693576 26760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e15f17435d04461b73f0a0756c245e3 [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:03.693930 26767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:13:03.693892 26759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e15f17435d04461b73f0a0756c245e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e15f17435d04461b73f0a0756c245e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e15f17435d04461b73f0a0756c245e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33657 } } }
I20260620 20:13:03.694038 26759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e15f17435d04461b73f0a0756c245e3 [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:03.696543 26767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:13:03.701809 26767 catalog_manager.cc:1347] Generated new cluster ID: ba5de45bdc6f45a49484ebf6eb237816
I20260620 20:13:03.701874 26767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:13:03.719254 26767 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:13:03.719594 26767 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:13:03.731042 26767 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e15f17435d04461b73f0a0756c245e3: Generated new TSK 0
I20260620 20:13:03.731756 26767 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:13:03.738782 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 26680
2026-06-20T20:13:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1331 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:13:03 dist-test-slave-l8h4 krb5kdc[26779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:13:03 dist-test-slave-l8h4 krb5kdc[26779](info): set up 2 sockets
Jun 20 20:13:03 dist-test-slave-l8h4 krb5kdc[26779](info): commencing operation
krb5kdc: starting...
W20260620 20:13:04.715342 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.958s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:13:04 dist-test-slave-l8h4 krb5kdc[26779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:13:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:13:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 20:13:04.837457 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43293
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:13:04.947084 26792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:13:04.947381 26792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:13:04.947439 26792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:13:04.951532 26792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:13:04.951608 26792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:13:04.951634 26792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:13:04.951776 26792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:13:04.951808 26792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:13:04.957489 26792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43293
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:13:04.958631 26792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:13:04.959731 26792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:13:04.968384 26797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:04.968393 26800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:04.968573 26792 server_base.cc:1034] running on GCE node
W20260620 20:13:04.968413 26798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:04.969730 26792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:13:04.971351 26792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:13:04.972558 26792 hybrid_clock.cc:648] HybridClock initialized: now 1781986384972535 us; error 36 us; skew 500 ppm
Jun 20 20:13:04 dist-test-slave-l8h4 krb5kdc[26779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986384, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:13:04.975457 26792 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 20:13:04.976410 26792 webserver.cc:460] Webserver started at http://127.0.0.1:44381/ using document root <none> and password file <none>
I20260620 20:13:04.976943 26792 fs_manager.cc:359] Metadata directory not provided
I20260620 20:13:04.977000 26792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:13:04.977180 26792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:13:04.979430 26792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "8bb90ed6cd824937a587336b09afcab6"
format_stamp: "Formatted at 2026-06-20 20:13:04 on dist-test-slave-l8h4"
server_key: "5a2dd08360cad29b887031bb1bbc9981"
server_key_iv: "e0dacdbe513475f1ec82c730bbe08fb1"
server_key_version: "encryptionkey@0"
I20260620 20:13:04.979949 26792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "8bb90ed6cd824937a587336b09afcab6"
format_stamp: "Formatted at 2026-06-20 20:13:04 on dist-test-slave-l8h4"
server_key: "5a2dd08360cad29b887031bb1bbc9981"
server_key_iv: "e0dacdbe513475f1ec82c730bbe08fb1"
server_key_version: "encryptionkey@0"
I20260620 20:13:04.983641 26792 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 20:13:04.986388 26807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:13:04.987730 26792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:13:04.987833 26792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "8bb90ed6cd824937a587336b09afcab6"
format_stamp: "Formatted at 2026-06-20 20:13:04 on dist-test-slave-l8h4"
server_key: "5a2dd08360cad29b887031bb1bbc9981"
server_key_iv: "e0dacdbe513475f1ec82c730bbe08fb1"
server_key_version: "encryptionkey@0"
I20260620 20:13:04.987946 26792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:13:05.000139 26792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:13:05.003443 26792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:13:05.003615 26792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:13:05.011430 26792 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43293
I20260620 20:13:05.011462 26859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43293 every 8 connection(s)
I20260620 20:13:05.013015 26792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:13:05.013927 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 26792
I20260620 20:13:05.014078 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:13:05.014458 5486 external_mini_cluster.cc:1428] Setting key 7007faa94ae0f8b1a25a1b913196b3ab
I20260620 20:13:05.017946 26860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:13:05.025439 26860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bb90ed6cd824937a587336b09afcab6: Bootstrap starting.
Jun 20 20:13:05 dist-test-slave-l8h4 krb5kdc[26779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 20:13:05.028007 26860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bb90ed6cd824937a587336b09afcab6: Neither blocks nor log segments found. Creating new log.
I20260620 20:13:05.028872 26860 log.cc:824] T 00000000000000000000000000000000 P 8bb90ed6cd824937a587336b09afcab6: Log is configured to *not* fsync() on all Append() calls
I20260620 20:13:05.031301 26860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bb90ed6cd824937a587336b09afcab6: No bootstrap required, opened a new log
I20260620 20:13:05.032771 26863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:05.016344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:48687 (local address 127.0.0.1:43293)
0620 20:13:05.016883 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:05.016908 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:05.016965 (+ 57us) server_negotiation.cc:407] Connection header received
0620 20:13:05.018065 (+ 1100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:05.018097 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:05.018728 (+ 631us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:05.019232 (+ 504us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:05.020373 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:05.021961 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:05.023186 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:05.023866 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:05.024035 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:05.027159 (+ 3124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:05.027166 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:05.027182 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:05.027218 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:05.029308 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:05.029941 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:05.029947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:05.029953 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:05.030090 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:05.030434 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:05.030437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:05.030439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:05.030928 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:05.031133 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:05.031368 (+ 235us) server_negotiation.cc:299] Negotiation successful
0620 20:13:05.031620 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":127,"threads_started":1}
I20260620 20:13:05.034958 26860 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8bb90ed6cd824937a587336b09afcab6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb90ed6cd824937a587336b09afcab6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43293 } }
I20260620 20:13:05.035197 26860 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8bb90ed6cd824937a587336b09afcab6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:13:05.035257 26860 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8bb90ed6cd824937a587336b09afcab6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bb90ed6cd824937a587336b09afcab6, State: Initialized, Role: FOLLOWER
I20260620 20:13:05.035674 26860 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bb90ed6cd824937a587336b09afcab6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb90ed6cd824937a587336b09afcab6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43293 } }
I20260620 20:13:05.035727 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:13:05.035797 26860 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8bb90ed6cd824937a587336b09afcab6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:13:05.035854 26860 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8bb90ed6cd824937a587336b09afcab6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:13:05.035948 26860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8bb90ed6cd824937a587336b09afcab6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:13:05.036787 26860 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8bb90ed6cd824937a587336b09afcab6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb90ed6cd824937a587336b09afcab6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43293 } }
I20260620 20:13:05.037196 26860 leader_election.cc:302] T 00000000000000000000000000000000 P 8bb90ed6cd824937a587336b09afcab6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8bb90ed6cd824937a587336b09afcab6; no voters:
I20260620 20:13:05.037509 26860 leader_election.cc:288] T 00000000000000000000000000000000 P 8bb90ed6cd824937a587336b09afcab6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:13:05.037624 26869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8bb90ed6cd824937a587336b09afcab6 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:13:05.037840 26869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8bb90ed6cd824937a587336b09afcab6 [term 1 LEADER]: Becoming Leader. State: Replica: 8bb90ed6cd824937a587336b09afcab6, State: Running, Role: LEADER
I20260620 20:13:05.038225 26869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8bb90ed6cd824937a587336b09afcab6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb90ed6cd824937a587336b09afcab6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43293 } }
I20260620 20:13:05.039026 26860 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8bb90ed6cd824937a587336b09afcab6 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:13:05.039330 26869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bb90ed6cd824937a587336b09afcab6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bb90ed6cd824937a587336b09afcab6. Latest consensus state: current_term: 1 leader_uuid: "8bb90ed6cd824937a587336b09afcab6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb90ed6cd824937a587336b09afcab6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43293 } } }
I20260620 20:13:05.039512 26869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bb90ed6cd824937a587336b09afcab6 [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:05.039729 26869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bb90ed6cd824937a587336b09afcab6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8bb90ed6cd824937a587336b09afcab6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb90ed6cd824937a587336b09afcab6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43293 } } }
I20260620 20:13:05.039842 26869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bb90ed6cd824937a587336b09afcab6 [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:05.040149 26874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:13:05.043015 26874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:13:05.046633 26863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:05.037711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:47777 (local address 127.0.0.1:43293)
0620 20:13:05.037818 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:05.037822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:05.038024 (+ 202us) server_negotiation.cc:407] Connection header received
0620 20:13:05.038184 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:05.038187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:05.038240 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:05.038329 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:05.039092 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:05.039843 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:05.040996 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:05.041312 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:05.041412 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:05.043144 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:05.043148 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:05.043151 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:05.043174 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:05.044773 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:05.045311 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:05.045316 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:05.045318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:05.045405 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:05.045741 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:05.045744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:05.045746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:05.045949 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:05.046090 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:05.046389 (+ 299us) server_negotiation.cc:299] Negotiation successful
0620 20:13:05.046485 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 20:13:05.048890 26874 catalog_manager.cc:1347] Generated new cluster ID: 0f1abd79ec994ec8bcd2754ca67b4839
I20260620 20:13:05.048974 26874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:13:05.063326 26874 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:13:05.064200 26874 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:13:05.070693 26874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8bb90ed6cd824937a587336b09afcab6: Generated new TSK 0
I20260620 20:13:05.071318 26874 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:13:05.077414 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 26792
W20260620 20:13:05.082161 26777 connection.cc:535] client connection to 127.0.0.1:43293 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43293): Connection reset by peer (error 104)
2026-06-20T20:13:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1338 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:13:05 dist-test-slave-l8h4 krb5kdc[26891](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:13:05 dist-test-slave-l8h4 krb5kdc[26891](info): set up 2 sockets
Jun 20 20:13:05 dist-test-slave-l8h4 krb5kdc[26891](info): commencing operation
krb5kdc: starting...
W20260620 20:13:06.051824 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.956s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:13:06 dist-test-slave-l8h4 krb5kdc[26891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:13:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:13:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 20:13:06.178299 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36157
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36157
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:13:06.291115 26904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:13:06.291410 26904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:13:06.291477 26904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:13:06.295749 26904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:13:06.295816 26904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:13:06.295842 26904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:13:06.295866 26904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:13:06.295894 26904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:13:06.301422 26904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36157
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:13:06.302558 26904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:13:06.303561 26904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:13:06.312292 26910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:06.312304 26912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:06.312304 26909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:06.312633 26904 server_base.cc:1034] running on GCE node
I20260620 20:13:06.313436 26904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:13:06.314639 26904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:13:06.315841 26904 hybrid_clock.cc:648] HybridClock initialized: now 1781986386315755 us; error 99 us; skew 500 ppm
Jun 20 20:13:06 dist-test-slave-l8h4 krb5kdc[26891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986386, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:13:06.318714 26904 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 20:13:06.319604 26904 webserver.cc:460] Webserver started at http://127.0.0.1:39377/ using document root <none> and password file <none>
I20260620 20:13:06.320217 26904 fs_manager.cc:359] Metadata directory not provided
I20260620 20:13:06.320282 26904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:13:06.320454 26904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:13:06.322518 26904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "5131ecd903e7423bb02c15296597177d"
format_stamp: "Formatted at 2026-06-20 20:13:06 on dist-test-slave-l8h4"
server_key: "61c35914633404d3b0e91eb5fde1008f"
server_key_iv: "7c476dbba288bd46d6ece3f9fa75c830"
server_key_version: "encryptionkey@0"
I20260620 20:13:06.323016 26904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "5131ecd903e7423bb02c15296597177d"
format_stamp: "Formatted at 2026-06-20 20:13:06 on dist-test-slave-l8h4"
server_key: "61c35914633404d3b0e91eb5fde1008f"
server_key_iv: "7c476dbba288bd46d6ece3f9fa75c830"
server_key_version: "encryptionkey@0"
I20260620 20:13:06.327093 26904 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 20:13:06.329809 26919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:13:06.331139 26904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:13:06.331256 26904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "5131ecd903e7423bb02c15296597177d"
format_stamp: "Formatted at 2026-06-20 20:13:06 on dist-test-slave-l8h4"
server_key: "61c35914633404d3b0e91eb5fde1008f"
server_key_iv: "7c476dbba288bd46d6ece3f9fa75c830"
server_key_version: "encryptionkey@0"
I20260620 20:13:06.331375 26904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:13:06.346565 26904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:13:06.349979 26904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:13:06.350193 26904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:13:06.356913 26904 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36157
I20260620 20:13:06.356966 26971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36157 every 8 connection(s)
I20260620 20:13:06.357877 26904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:13:06.361625 26972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:13:06.365362 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 26904
I20260620 20:13:06.365475 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:13:06.365758 5486 external_mini_cluster.cc:1428] Setting key 4be9733e491e2ef99ac3349fd7cb2aa5
I20260620 20:13:06.367667 26972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5131ecd903e7423bb02c15296597177d: Bootstrap starting.
I20260620 20:13:06.369798 26972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5131ecd903e7423bb02c15296597177d: Neither blocks nor log segments found. Creating new log.
I20260620 20:13:06.370584 26972 log.cc:824] T 00000000000000000000000000000000 P 5131ecd903e7423bb02c15296597177d: Log is configured to *not* fsync() on all Append() calls
I20260620 20:13:06.373044 26972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5131ecd903e7423bb02c15296597177d: No bootstrap required, opened a new log
I20260620 20:13:06.376422 26975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:06.367116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:60751 (local address 127.0.0.1:36157)
0620 20:13:06.367615 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:06.367631 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:06.367668 (+ 37us) server_negotiation.cc:407] Connection header received
0620 20:13:06.368380 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:06.368398 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:06.368750 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:06.369058 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:06.369830 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:06.371242 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:06.372637 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:06.373328 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:06.373464 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:06.374403 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:06.374408 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:06.374421 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:13:06.374920 (+ 499us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:06.374980 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:13:06.375003 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:06.375118 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:06.375283 (+ 165us) server_negotiation.cc:299] Negotiation successful
0620 20:13:06.375412 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":142,"threads_started":1}
I20260620 20:13:06.377005 26972 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5131ecd903e7423bb02c15296597177d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5131ecd903e7423bb02c15296597177d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36157 } }
I20260620 20:13:06.377221 26972 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5131ecd903e7423bb02c15296597177d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:13:06.377298 26972 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5131ecd903e7423bb02c15296597177d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5131ecd903e7423bb02c15296597177d, State: Initialized, Role: FOLLOWER
I20260620 20:13:06.377753 26972 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5131ecd903e7423bb02c15296597177d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5131ecd903e7423bb02c15296597177d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36157 } }
I20260620 20:13:06.377878 26972 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5131ecd903e7423bb02c15296597177d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:13:06.377921 26972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5131ecd903e7423bb02c15296597177d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:13:06.377990 26972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5131ecd903e7423bb02c15296597177d [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:13:06.378978 26972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5131ecd903e7423bb02c15296597177d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5131ecd903e7423bb02c15296597177d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36157 } }
I20260620 20:13:06.379176 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:13:06.379315 26972 leader_election.cc:302] T 00000000000000000000000000000000 P 5131ecd903e7423bb02c15296597177d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5131ecd903e7423bb02c15296597177d; no voters:
I20260620 20:13:06.379637 26972 leader_election.cc:288] T 00000000000000000000000000000000 P 5131ecd903e7423bb02c15296597177d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:13:06.379760 26977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5131ecd903e7423bb02c15296597177d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:13:06.379981 26977 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5131ecd903e7423bb02c15296597177d [term 1 LEADER]: Becoming Leader. State: Replica: 5131ecd903e7423bb02c15296597177d, State: Running, Role: LEADER
I20260620 20:13:06.380316 26977 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5131ecd903e7423bb02c15296597177d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5131ecd903e7423bb02c15296597177d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36157 } }
I20260620 20:13:06.380972 26972 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5131ecd903e7423bb02c15296597177d [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:13:06.381482 26977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5131ecd903e7423bb02c15296597177d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5131ecd903e7423bb02c15296597177d. Latest consensus state: current_term: 1 leader_uuid: "5131ecd903e7423bb02c15296597177d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5131ecd903e7423bb02c15296597177d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36157 } } }
I20260620 20:13:06.381639 26977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5131ecd903e7423bb02c15296597177d [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:06.381888 26978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5131ecd903e7423bb02c15296597177d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5131ecd903e7423bb02c15296597177d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5131ecd903e7423bb02c15296597177d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36157 } } }
I20260620 20:13:06.381980 26978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5131ecd903e7423bb02c15296597177d [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:06.383203 26987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:13:06.386264 26987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:13:06.387243 26975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:06.381445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:33283 (local address 127.0.0.1:36157)
0620 20:13:06.381544 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:06.381548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:06.382425 (+ 877us) server_negotiation.cc:407] Connection header received
0620 20:13:06.382543 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:06.382546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:06.382596 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:06.382678 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:06.383090 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:06.383799 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:06.384964 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:06.385250 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:06.385471 (+ 221us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:06.386734 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:06.386738 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:06.386741 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:13:06.386757 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:06.386795 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:13:06.386809 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:06.386903 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:06.387072 (+ 169us) server_negotiation.cc:299] Negotiation successful
0620 20:13:06.387114 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 20:13:06.392169 26987 catalog_manager.cc:1347] Generated new cluster ID: 6e5348e84b454a4ca7d768a078736c5d
I20260620 20:13:06.392239 26987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:13:06.413692 26987 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:13:06.414708 26987 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:13:06.420976 26987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5131ecd903e7423bb02c15296597177d: Generated new TSK 0
I20260620 20:13:06.421651 26987 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:13:06.434329 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 26904
W20260620 20:13:06.439225 26889 connection.cc:535] client connection to 127.0.0.1:36157 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36157): Connection reset by peer (error 104)
2026-06-20T20:13:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1357 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:13:06 dist-test-slave-l8h4 krb5kdc[27003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:13:06 dist-test-slave-l8h4 krb5kdc[27003](info): set up 2 sockets
Jun 20 20:13:06 dist-test-slave-l8h4 krb5kdc[27003](info): commencing operation
krb5kdc: starting...
W20260620 20:13:07.406965 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.953s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:13:07 dist-test-slave-l8h4 krb5kdc[27003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:13:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:13:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 20:13:07.525715 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46295
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:13:07.634384 27016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:13:07.634677 27016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:13:07.634735 27016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:13:07.638789 27016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:13:07.638854 27016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:13:07.638881 27016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:13:07.638918 27016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:13:07.638947 27016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:13:07.644730 27016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46295
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:13:07.645823 27016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:13:07.646827 27016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:13:07.654944 27022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:07.654950 27024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:07.655084 27021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:07.655377 27016 server_base.cc:1034] running on GCE node
I20260620 20:13:07.656019 27016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:13:07.657169 27016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:13:07.658330 27016 hybrid_clock.cc:648] HybridClock initialized: now 1781986387658311 us; error 29 us; skew 500 ppm
Jun 20 20:13:07 dist-test-slave-l8h4 krb5kdc[27003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986387, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:13:07.665690 27016 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 20:13:07.666738 27016 webserver.cc:460] Webserver started at http://127.0.0.1:32961/ using document root <none> and password file <none>
I20260620 20:13:07.667277 27016 fs_manager.cc:359] Metadata directory not provided
I20260620 20:13:07.667341 27016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:13:07.667527 27016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:13:07.669548 27016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "10acc1b1c56a492da4786400397320fb"
format_stamp: "Formatted at 2026-06-20 20:13:07 on dist-test-slave-l8h4"
server_key: "d9775fda95c9df26f7d72adda927b6c4"
server_key_iv: "b2480eaaa0e9e2e25b83492f20b72c1d"
server_key_version: "encryptionkey@0"
I20260620 20:13:07.670087 27016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "10acc1b1c56a492da4786400397320fb"
format_stamp: "Formatted at 2026-06-20 20:13:07 on dist-test-slave-l8h4"
server_key: "d9775fda95c9df26f7d72adda927b6c4"
server_key_iv: "b2480eaaa0e9e2e25b83492f20b72c1d"
server_key_version: "encryptionkey@0"
I20260620 20:13:07.673768 27016 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:13:07.676537 27031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:13:07.677774 27016 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 20:13:07.677896 27016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "10acc1b1c56a492da4786400397320fb"
format_stamp: "Formatted at 2026-06-20 20:13:07 on dist-test-slave-l8h4"
server_key: "d9775fda95c9df26f7d72adda927b6c4"
server_key_iv: "b2480eaaa0e9e2e25b83492f20b72c1d"
server_key_version: "encryptionkey@0"
I20260620 20:13:07.677999 27016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:13:07.682617 27016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:13:07.685459 27016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:13:07.685629 27016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:13:07.692301 27016 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46295
I20260620 20:13:07.692328 27083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46295 every 8 connection(s)
I20260620 20:13:07.693212 27016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:13:07.695906 27084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:13:07.701243 27084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb: Bootstrap starting.
I20260620 20:13:07.702798 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 27016
I20260620 20:13:07.702912 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:13:07.703173 5486 external_mini_cluster.cc:1428] Setting key f35d75f0bfe3f50cddfd00f7830d9cee
I20260620 20:13:07.703722 27084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb: Neither blocks nor log segments found. Creating new log.
I20260620 20:13:07.704514 27084 log.cc:824] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb: Log is configured to *not* fsync() on all Append() calls
I20260620 20:13:07.706777 27084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb: No bootstrap required, opened a new log
I20260620 20:13:07.708340 27087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:07.704267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:38535 (local address 127.0.0.1:46295)
0620 20:13:07.704786 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:07.704804 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:07.704848 (+ 44us) server_negotiation.cc:407] Connection header received
0620 20:13:07.705704 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:07.705726 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:07.706051 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:07.706419 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:07.707060 (+ 641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:07.707065 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:07.707078 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:13:07.707089 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:07.707151 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:13:07.707177 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:07.707279 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:07.707342 (+ 63us) server_negotiation.cc:299] Negotiation successful
0620 20:13:07.707486 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":141,"threads_started":1}
I20260620 20:13:07.710579 27084 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10acc1b1c56a492da4786400397320fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46295 } }
I20260620 20:13:07.710786 27084 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:13:07.710827 27084 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10acc1b1c56a492da4786400397320fb, State: Initialized, Role: FOLLOWER
I20260620 20:13:07.711107 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:13:07.711272 27084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10acc1b1c56a492da4786400397320fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46295 } }
I20260620 20:13:07.711413 27084 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:13:07.711467 27084 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:13:07.711552 27084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:13:07.712698 27084 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10acc1b1c56a492da4786400397320fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46295 } }
I20260620 20:13:07.713056 27084 leader_election.cc:302] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10acc1b1c56a492da4786400397320fb; no voters:
I20260620 20:13:07.713342 27084 leader_election.cc:288] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:13:07.713418 27093 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:13:07.713707 27093 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb [term 1 LEADER]: Becoming Leader. State: Replica: 10acc1b1c56a492da4786400397320fb, State: Running, Role: LEADER
I20260620 20:13:07.714046 27093 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10acc1b1c56a492da4786400397320fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46295 } }
I20260620 20:13:07.714648 27084 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:13:07.715387 27087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:07.713329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:57379 (local address 127.0.0.1:46295)
0620 20:13:07.713459 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:07.713463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:07.713696 (+ 233us) server_negotiation.cc:407] Connection header received
0620 20:13:07.713850 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:07.713853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:07.713891 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:07.713965 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:07.714761 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:07.714765 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:07.714767 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:13:07.714778 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:07.714806 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:13:07.714818 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:07.714885 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:07.715222 (+ 337us) server_negotiation.cc:299] Negotiation successful
0620 20:13:07.715262 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 20:13:07.715947 27095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10acc1b1c56a492da4786400397320fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10acc1b1c56a492da4786400397320fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46295 } } }
I20260620 20:13:07.715988 27096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10acc1b1c56a492da4786400397320fb. Latest consensus state: current_term: 1 leader_uuid: "10acc1b1c56a492da4786400397320fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10acc1b1c56a492da4786400397320fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46295 } } }
I20260620 20:13:07.716105 27095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:07.716109 27096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb [sys.catalog]: This master's current role is: LEADER
W20260620 20:13:07.719297 27109 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 20:13:07.719388 27109 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 20:13:07.719493 27107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:13:07.720183 27107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:13:07.725576 27107 catalog_manager.cc:1347] Generated new cluster ID: 3b2065a952a64f7cbbd6b3a7f81f1275
I20260620 20:13:07.725653 27107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:13:07.740001 27107 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:13:07.740309 27107 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:13:07.746225 27107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10acc1b1c56a492da4786400397320fb: Generated new TSK 0
I20260620 20:13:07.746879 27107 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:13:07.754837 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 27016
2026-06-20T20:13:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1319 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:13:07 dist-test-slave-l8h4 krb5kdc[27115](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:13:07 dist-test-slave-l8h4 krb5kdc[27115](info): set up 2 sockets
Jun 20 20:13:07 dist-test-slave-l8h4 krb5kdc[27115](info): commencing operation
krb5kdc: starting...
W20260620 20:13:08.715377 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.942s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:13:08 dist-test-slave-l8h4 krb5kdc[27115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:13:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:13:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:13:08.838703 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:32829
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:44835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:32829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:13:08.949162 27128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:13:08.949445 27128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:13:08.949488 27128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:13:08.953656 27128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:13:08.953732 27128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:13:08.953756 27128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:13:08.953773 27128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:13:08.953785 27128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:13:08.959173 27128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:44835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:32829
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:32829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:13:08.960217 27128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:13:08.961195 27128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:13:08.969331 27134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:08.969358 27133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:08.969326 27136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:08.969596 27128 server_base.cc:1034] running on GCE node
I20260620 20:13:08.970253 27128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:13:08.971279 27128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:13:08.972432 27128 hybrid_clock.cc:648] HybridClock initialized: now 1781986388972418 us; error 26 us; skew 500 ppm
Jun 20 20:13:08 dist-test-slave-l8h4 krb5kdc[27115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986388, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:13:08.975052 27128 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:13:08.976630 27128 webserver.cc:460] Webserver started at http://127.5.91.190:33285/ using document root <none> and password file <none>
I20260620 20:13:08.977161 27128 fs_manager.cc:359] Metadata directory not provided
I20260620 20:13:08.977226 27128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:13:08.977409 27128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:13:08.979450 27128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "65d5b72bee0f47188a1b47d23e31294e"
format_stamp: "Formatted at 2026-06-20 20:13:08 on dist-test-slave-l8h4"
server_key: "3a7352ed5d95fea1d9e8d796829278ce"
server_key_iv: "3e9f5f9dbea71d4cb79ae2d3635b6ab9"
server_key_version: "encryptionkey@0"
I20260620 20:13:08.979981 27128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "65d5b72bee0f47188a1b47d23e31294e"
format_stamp: "Formatted at 2026-06-20 20:13:08 on dist-test-slave-l8h4"
server_key: "3a7352ed5d95fea1d9e8d796829278ce"
server_key_iv: "3e9f5f9dbea71d4cb79ae2d3635b6ab9"
server_key_version: "encryptionkey@0"
I20260620 20:13:08.984019 27128 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 20:13:08.986797 27143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:13:08.988042 27128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:13:08.988170 27128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "65d5b72bee0f47188a1b47d23e31294e"
format_stamp: "Formatted at 2026-06-20 20:13:08 on dist-test-slave-l8h4"
server_key: "3a7352ed5d95fea1d9e8d796829278ce"
server_key_iv: "3e9f5f9dbea71d4cb79ae2d3635b6ab9"
server_key_version: "encryptionkey@0"
I20260620 20:13:08.988270 27128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:13:09.003404 27128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:13:09.006814 27128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:13:09.007009 27128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:13:09.014174 27128 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:32829
I20260620 20:13:09.014215 27195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:32829 every 8 connection(s)
I20260620 20:13:09.015144 27128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:13:09.016168 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 27128
I20260620 20:13:09.016288 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:13:09.016553 5486 external_mini_cluster.cc:1428] Setting key 105978c777bfd48bf3c2fdbca8b852e4
I20260620 20:13:09.018174 27196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:13:09.024531 27196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d5b72bee0f47188a1b47d23e31294e: Bootstrap starting.
Jun 20 20:13:09 dist-test-slave-l8h4 krb5kdc[27115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:13:09.027168 27196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65d5b72bee0f47188a1b47d23e31294e: Neither blocks nor log segments found. Creating new log.
I20260620 20:13:09.028483 27196 log.cc:824] T 00000000000000000000000000000000 P 65d5b72bee0f47188a1b47d23e31294e: Log is configured to *not* fsync() on all Append() calls
I20260620 20:13:09.030819 27196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d5b72bee0f47188a1b47d23e31294e: No bootstrap required, opened a new log
I20260620 20:13:09.032925 27199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:09.017907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36717 (local address 127.5.91.190:32829)
0620 20:13:09.018325 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:09.018343 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:09.018529 (+ 186us) server_negotiation.cc:407] Connection header received
0620 20:13:09.019373 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:09.019392 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:09.019795 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:09.020147 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:09.020981 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:09.022449 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:09.023337 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:09.024015 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:09.024168 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:09.027815 (+ 3647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:09.027820 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:09.027834 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:09.027863 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:09.029861 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:09.030545 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:09.030550 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:09.030554 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:09.030660 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:09.031012 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:09.031021 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:09.031022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:09.031410 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:09.031612 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:09.031984 (+ 372us) server_negotiation.cc:299] Negotiation successful
0620 20:13:09.032223 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":109,"threads_started":1}
I20260620 20:13:09.034511 27196 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65d5b72bee0f47188a1b47d23e31294e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d5b72bee0f47188a1b47d23e31294e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 32829 } }
I20260620 20:13:09.034756 27196 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65d5b72bee0f47188a1b47d23e31294e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:13:09.034821 27196 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65d5b72bee0f47188a1b47d23e31294e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65d5b72bee0f47188a1b47d23e31294e, State: Initialized, Role: FOLLOWER
I20260620 20:13:09.035266 27196 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65d5b72bee0f47188a1b47d23e31294e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d5b72bee0f47188a1b47d23e31294e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 32829 } }
I20260620 20:13:09.035398 27196 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65d5b72bee0f47188a1b47d23e31294e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:13:09.035444 27196 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65d5b72bee0f47188a1b47d23e31294e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:13:09.035537 27196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65d5b72bee0f47188a1b47d23e31294e [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:13:09.035748 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:13:09.036671 27196 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65d5b72bee0f47188a1b47d23e31294e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d5b72bee0f47188a1b47d23e31294e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 32829 } }
I20260620 20:13:09.037042 27196 leader_election.cc:302] T 00000000000000000000000000000000 P 65d5b72bee0f47188a1b47d23e31294e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65d5b72bee0f47188a1b47d23e31294e; no voters:
I20260620 20:13:09.037468 27196 leader_election.cc:288] T 00000000000000000000000000000000 P 65d5b72bee0f47188a1b47d23e31294e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:13:09.037905 27205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65d5b72bee0f47188a1b47d23e31294e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:13:09.038199 27205 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65d5b72bee0f47188a1b47d23e31294e [term 1 LEADER]: Becoming Leader. State: Replica: 65d5b72bee0f47188a1b47d23e31294e, State: Running, Role: LEADER
I20260620 20:13:09.038606 27205 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65d5b72bee0f47188a1b47d23e31294e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d5b72bee0f47188a1b47d23e31294e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 32829 } }
I20260620 20:13:09.039254 27196 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65d5b72bee0f47188a1b47d23e31294e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:13:09.040359 27208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65d5b72bee0f47188a1b47d23e31294e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65d5b72bee0f47188a1b47d23e31294e. Latest consensus state: current_term: 1 leader_uuid: "65d5b72bee0f47188a1b47d23e31294e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d5b72bee0f47188a1b47d23e31294e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 32829 } } }
I20260620 20:13:09.040510 27208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65d5b72bee0f47188a1b47d23e31294e [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:09.040889 27206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65d5b72bee0f47188a1b47d23e31294e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65d5b72bee0f47188a1b47d23e31294e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d5b72bee0f47188a1b47d23e31294e" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 32829 } } }
I20260620 20:13:09.041049 27206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65d5b72bee0f47188a1b47d23e31294e [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:09.042193 27215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:13:09.045473 27215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:13:09.047947 27199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:09.038093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57849 (local address 127.5.91.190:32829)
0620 20:13:09.038314 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:09.038318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:09.038478 (+ 160us) server_negotiation.cc:407] Connection header received
0620 20:13:09.038639 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:09.038642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:09.038691 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:09.038756 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:09.039636 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:09.040706 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:09.041871 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:09.042228 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:09.042322 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:09.044027 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:09.044031 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:09.044034 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:09.044059 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:09.045682 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:09.046275 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:09.046279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:09.046281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:09.046362 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:09.046728 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:09.046731 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:09.046733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:09.046941 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:09.047071 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:09.047686 (+ 615us) server_negotiation.cc:299] Negotiation successful
0620 20:13:09.047784 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260620 20:13:09.052153 27215 catalog_manager.cc:1347] Generated new cluster ID: f3943b1e88be4c6f9435b63793358d09
I20260620 20:13:09.052218 27215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:13:09.062619 27215 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:13:09.063903 27215 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:13:09.068837 27215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65d5b72bee0f47188a1b47d23e31294e: Generated new TSK 0
I20260620 20:13:09.069672 27215 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:13:09.076643 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 27128
W20260620 20:13:09.081255 27113 connection.cc:535] client connection to 127.5.91.190:32829 recv error: Network error: failed to read from TLS socket (remote: 127.5.91.190:32829): Connection reset by peer (error 104)
2026-06-20T20:13:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1321 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:13:09 dist-test-slave-l8h4 krb5kdc[27227](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:13:09 dist-test-slave-l8h4 krb5kdc[27227](info): set up 2 sockets
Jun 20 20:13:09 dist-test-slave-l8h4 krb5kdc[27227](info): commencing operation
krb5kdc: starting...
W20260620 20:13:10.071089 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.975s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:13:10 dist-test-slave-l8h4 krb5kdc[27227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:13:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:13:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:13:10.202564 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:34369
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:44385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:34369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:13:10.324227 27240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:13:10.324571 27240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:13:10.324635 27240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:13:10.329725 27240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:13:10.329821 27240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:13:10.329854 27240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:13:10.329875 27240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:13:10.329895 27240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:13:10.335573 27240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:44385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:34369
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:34369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:13:10.336707 27240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:13:10.337857 27240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:13:10.347556 27246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:10.347990 27245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:10.348067 27240 server_base.cc:1034] running on GCE node
W20260620 20:13:10.348172 27248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:10.348793 27240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:13:10.349987 27240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:13:10.351229 27240 hybrid_clock.cc:648] HybridClock initialized: now 1781986390351203 us; error 40 us; skew 500 ppm
Jun 20 20:13:10 dist-test-slave-l8h4 krb5kdc[27227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986390, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:13:10.354518 27240 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:13:10.355705 27240 webserver.cc:460] Webserver started at http://127.5.91.190:41223/ using document root <none> and password file <none>
I20260620 20:13:10.356282 27240 fs_manager.cc:359] Metadata directory not provided
I20260620 20:13:10.356364 27240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:13:10.356572 27240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:13:10.358760 27240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "a36b1ce4a6a84022abe9e3f64e717f84"
format_stamp: "Formatted at 2026-06-20 20:13:10 on dist-test-slave-l8h4"
server_key: "7eb95973182f10091e33704135cb7e91"
server_key_iv: "336eec13d9f2e7ffa09578cb392795d5"
server_key_version: "encryptionkey@0"
I20260620 20:13:10.359336 27240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "a36b1ce4a6a84022abe9e3f64e717f84"
format_stamp: "Formatted at 2026-06-20 20:13:10 on dist-test-slave-l8h4"
server_key: "7eb95973182f10091e33704135cb7e91"
server_key_iv: "336eec13d9f2e7ffa09578cb392795d5"
server_key_version: "encryptionkey@0"
I20260620 20:13:10.363561 27240 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260620 20:13:10.367189 27255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:13:10.368917 27240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 20:13:10.369056 27240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "a36b1ce4a6a84022abe9e3f64e717f84"
format_stamp: "Formatted at 2026-06-20 20:13:10 on dist-test-slave-l8h4"
server_key: "7eb95973182f10091e33704135cb7e91"
server_key_iv: "336eec13d9f2e7ffa09578cb392795d5"
server_key_version: "encryptionkey@0"
I20260620 20:13:10.369206 27240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:13:10.382790 27240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:13:10.386778 27240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:13:10.386955 27240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:13:10.394379 27240 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:34369
I20260620 20:13:10.394450 27307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:34369 every 8 connection(s)
I20260620 20:13:10.395352 27240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:13:10.398288 27308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:13:10.400054 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 27240
I20260620 20:13:10.400185 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:13:10.400470 5486 external_mini_cluster.cc:1428] Setting key 5493735932053a2334195a6b1fe154bb
I20260620 20:13:10.404642 27308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a36b1ce4a6a84022abe9e3f64e717f84: Bootstrap starting.
I20260620 20:13:10.406816 27308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a36b1ce4a6a84022abe9e3f64e717f84: Neither blocks nor log segments found. Creating new log.
I20260620 20:13:10.407546 27308 log.cc:824] T 00000000000000000000000000000000 P a36b1ce4a6a84022abe9e3f64e717f84: Log is configured to *not* fsync() on all Append() calls
I20260620 20:13:10.409300 27311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:10.401918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43493 (local address 127.5.91.190:34369)
0620 20:13:10.402381 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:10.402400 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:10.402437 (+ 37us) server_negotiation.cc:407] Connection header received
0620 20:13:10.403089 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:10.403113 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:10.403442 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:10.403738 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:10.404158 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:10.405151 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:10.405984 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:10.406473 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:10.406621 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:10.407410 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:10.407414 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:10.407428 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:13:10.408008 (+ 580us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:10.408076 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:13:10.408109 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:10.408255 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:10.408425 (+ 170us) server_negotiation.cc:299] Negotiation successful
0620 20:13:10.408565 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":159,"threads_started":1}
I20260620 20:13:10.409582 27308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a36b1ce4a6a84022abe9e3f64e717f84: No bootstrap required, opened a new log
I20260620 20:13:10.412274 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:13:10.413782 27308 raft_consensus.cc:348] T 00000000000000000000000000000000 P a36b1ce4a6a84022abe9e3f64e717f84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a36b1ce4a6a84022abe9e3f64e717f84" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 34369 } }
I20260620 20:13:10.414075 27308 raft_consensus.cc:374] T 00000000000000000000000000000000 P a36b1ce4a6a84022abe9e3f64e717f84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:13:10.414168 27308 raft_consensus.cc:729] T 00000000000000000000000000000000 P a36b1ce4a6a84022abe9e3f64e717f84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a36b1ce4a6a84022abe9e3f64e717f84, State: Initialized, Role: FOLLOWER
I20260620 20:13:10.414629 27308 consensus_queue.cc:260] T 00000000000000000000000000000000 P a36b1ce4a6a84022abe9e3f64e717f84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a36b1ce4a6a84022abe9e3f64e717f84" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 34369 } }
I20260620 20:13:10.414772 27308 raft_consensus.cc:388] T 00000000000000000000000000000000 P a36b1ce4a6a84022abe9e3f64e717f84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:13:10.414834 27308 raft_consensus.cc:482] T 00000000000000000000000000000000 P a36b1ce4a6a84022abe9e3f64e717f84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:13:10.414928 27308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a36b1ce4a6a84022abe9e3f64e717f84 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:13:10.416095 27308 raft_consensus.cc:504] T 00000000000000000000000000000000 P a36b1ce4a6a84022abe9e3f64e717f84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a36b1ce4a6a84022abe9e3f64e717f84" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 34369 } }
I20260620 20:13:10.416455 27308 leader_election.cc:302] T 00000000000000000000000000000000 P a36b1ce4a6a84022abe9e3f64e717f84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a36b1ce4a6a84022abe9e3f64e717f84; no voters:
I20260620 20:13:10.416736 27308 leader_election.cc:288] T 00000000000000000000000000000000 P a36b1ce4a6a84022abe9e3f64e717f84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:13:10.416841 27318 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a36b1ce4a6a84022abe9e3f64e717f84 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:13:10.417232 27318 raft_consensus.cc:686] T 00000000000000000000000000000000 P a36b1ce4a6a84022abe9e3f64e717f84 [term 1 LEADER]: Becoming Leader. State: Replica: a36b1ce4a6a84022abe9e3f64e717f84, State: Running, Role: LEADER
I20260620 20:13:10.417539 27318 consensus_queue.cc:237] T 00000000000000000000000000000000 P a36b1ce4a6a84022abe9e3f64e717f84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a36b1ce4a6a84022abe9e3f64e717f84" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 34369 } }
I20260620 20:13:10.418398 27308 sys_catalog.cc:564] T 00000000000000000000000000000000 P a36b1ce4a6a84022abe9e3f64e717f84 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:13:10.419179 27319 sys_catalog.cc:455] T 00000000000000000000000000000000 P a36b1ce4a6a84022abe9e3f64e717f84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a36b1ce4a6a84022abe9e3f64e717f84. Latest consensus state: current_term: 1 leader_uuid: "a36b1ce4a6a84022abe9e3f64e717f84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a36b1ce4a6a84022abe9e3f64e717f84" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 34369 } } }
I20260620 20:13:10.419281 27311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:10.414586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48117 (local address 127.5.91.190:34369)
0620 20:13:10.414709 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:10.414713 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:10.414936 (+ 223us) server_negotiation.cc:407] Connection header received
0620 20:13:10.415071 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:10.415074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:10.415132 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:10.415219 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:10.415497 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:10.416246 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:10.417216 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:10.417576 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:10.417689 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:10.418622 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:10.418625 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:10.418628 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:13:10.418641 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:10.418668 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:13:10.418681 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:10.418757 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:10.419074 (+ 317us) server_negotiation.cc:299] Negotiation successful
0620 20:13:10.419113 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 20:13:10.419312 27319 sys_catalog.cc:458] T 00000000000000000000000000000000 P a36b1ce4a6a84022abe9e3f64e717f84 [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:10.419667 27327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:13:10.419773 27320 sys_catalog.cc:455] T 00000000000000000000000000000000 P a36b1ce4a6a84022abe9e3f64e717f84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a36b1ce4a6a84022abe9e3f64e717f84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a36b1ce4a6a84022abe9e3f64e717f84" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 34369 } } }
I20260620 20:13:10.419842 27320 sys_catalog.cc:458] T 00000000000000000000000000000000 P a36b1ce4a6a84022abe9e3f64e717f84 [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:10.423054 27327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:13:10.428752 27327 catalog_manager.cc:1347] Generated new cluster ID: 7c6cffd416e340369f04db73473a2b33
I20260620 20:13:10.428830 27327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:13:10.460711 27327 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:13:10.462038 27327 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:13:10.471047 27327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a36b1ce4a6a84022abe9e3f64e717f84: Generated new TSK 0
I20260620 20:13:10.471896 27327 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:13:10.479571 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 27240
W20260620 20:13:10.484995 27225 connection.cc:535] client connection to 127.5.91.190:34369 recv error: Network error: failed to read from TLS socket (remote: 127.5.91.190:34369): Connection reset by peer (error 104)
2026-06-20T20:13:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1404 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:13:10 dist-test-slave-l8h4 krb5kdc[27339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:13:10 dist-test-slave-l8h4 krb5kdc[27339](info): set up 2 sockets
Jun 20 20:13:10 dist-test-slave-l8h4 krb5kdc[27339](info): commencing operation
krb5kdc: starting...
W20260620 20:13:11.458926 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.959s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:13:11 dist-test-slave-l8h4 krb5kdc[27339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:13:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:13:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:13:11.579314 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:41929
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:37547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:41929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:13:11.687518 27352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:13:11.687799 27352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:13:11.687842 27352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:13:11.692202 27352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:13:11.692282 27352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:13:11.692302 27352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:13:11.692318 27352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:13:11.692332 27352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:13:11.697844 27352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:37547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:41929
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:41929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:13:11.699033 27352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:13:11.700011 27352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:13:11.708551 27360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:11.708551 27357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:11.708712 27352 server_base.cc:1034] running on GCE node
W20260620 20:13:11.708576 27358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:11.709390 27352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:13:11.710486 27352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:13:11.711660 27352 hybrid_clock.cc:648] HybridClock initialized: now 1781986391711635 us; error 40 us; skew 500 ppm
Jun 20 20:13:11 dist-test-slave-l8h4 krb5kdc[27339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986391, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:13:11.714429 27352 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:13:11.715448 27352 webserver.cc:460] Webserver started at http://127.5.91.190:44561/ using document root <none> and password file <none>
I20260620 20:13:11.716193 27352 fs_manager.cc:359] Metadata directory not provided
I20260620 20:13:11.716277 27352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:13:11.716477 27352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:13:11.718644 27352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "59e67743e1fc4cb9a983af828cbdc40c"
format_stamp: "Formatted at 2026-06-20 20:13:11 on dist-test-slave-l8h4"
server_key: "1a29077b43c20981eaa78a8f8f406655"
server_key_iv: "6cc99b70456c326dc242ef80956bde88"
server_key_version: "encryptionkey@0"
I20260620 20:13:11.719184 27352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "59e67743e1fc4cb9a983af828cbdc40c"
format_stamp: "Formatted at 2026-06-20 20:13:11 on dist-test-slave-l8h4"
server_key: "1a29077b43c20981eaa78a8f8f406655"
server_key_iv: "6cc99b70456c326dc242ef80956bde88"
server_key_version: "encryptionkey@0"
I20260620 20:13:11.723448 27352 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260620 20:13:11.726224 27367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:13:11.727798 27352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:13:11.727995 27352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "59e67743e1fc4cb9a983af828cbdc40c"
format_stamp: "Formatted at 2026-06-20 20:13:11 on dist-test-slave-l8h4"
server_key: "1a29077b43c20981eaa78a8f8f406655"
server_key_iv: "6cc99b70456c326dc242ef80956bde88"
server_key_version: "encryptionkey@0"
I20260620 20:13:11.728130 27352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:13:11.733117 27352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:13:11.736778 27352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:13:11.736969 27352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:13:11.744351 27352 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:41929
I20260620 20:13:11.744405 27419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:41929 every 8 connection(s)
I20260620 20:13:11.745384 27352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:13:11.749048 27420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:13:11.755463 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 27352
I20260620 20:13:11.755589 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:13:11.755587 27420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59e67743e1fc4cb9a983af828cbdc40c: Bootstrap starting.
I20260620 20:13:11.755852 5486 external_mini_cluster.cc:1428] Setting key 30032d5169e823abc08da0a5a56a4c7f
I20260620 20:13:11.758285 27420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59e67743e1fc4cb9a983af828cbdc40c: Neither blocks nor log segments found. Creating new log.
I20260620 20:13:11.758992 27420 log.cc:824] T 00000000000000000000000000000000 P 59e67743e1fc4cb9a983af828cbdc40c: Log is configured to *not* fsync() on all Append() calls
I20260620 20:13:11.761364 27420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59e67743e1fc4cb9a983af828cbdc40c: No bootstrap required, opened a new log
I20260620 20:13:11.761540 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:11.757201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33819 (local address 127.5.91.190:41929)
0620 20:13:11.757696 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:11.757711 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:11.757748 (+ 37us) server_negotiation.cc:407] Connection header received
0620 20:13:11.758535 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:11.758554 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:11.758839 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:11.759156 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:11.759915 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:11.759921 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:11.759936 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:13:11.759950 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:11.760017 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:13:11.760047 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:11.760161 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:11.760340 (+ 179us) server_negotiation.cc:299] Negotiation successful
0620 20:13:11.760660 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":140,"threads_started":1}
I20260620 20:13:11.764703 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:13:11.765167 27420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59e67743e1fc4cb9a983af828cbdc40c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e67743e1fc4cb9a983af828cbdc40c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41929 } }
I20260620 20:13:11.765391 27420 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59e67743e1fc4cb9a983af828cbdc40c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:13:11.765445 27420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59e67743e1fc4cb9a983af828cbdc40c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59e67743e1fc4cb9a983af828cbdc40c, State: Initialized, Role: FOLLOWER
I20260620 20:13:11.766111 27420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59e67743e1fc4cb9a983af828cbdc40c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e67743e1fc4cb9a983af828cbdc40c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41929 } }
I20260620 20:13:11.766319 27420 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59e67743e1fc4cb9a983af828cbdc40c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:13:11.766386 27420 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59e67743e1fc4cb9a983af828cbdc40c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:13:11.766480 27420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59e67743e1fc4cb9a983af828cbdc40c [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:13:11.767531 27420 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59e67743e1fc4cb9a983af828cbdc40c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e67743e1fc4cb9a983af828cbdc40c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41929 } }
I20260620 20:13:11.768394 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:11.766828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51113 (local address 127.5.91.190:41929)
0620 20:13:11.766939 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:11.766944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:11.767099 (+ 155us) server_negotiation.cc:407] Connection header received
0620 20:13:11.767212 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:11.767215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:11.767250 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:11.767315 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:11.767963 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:11.767966 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:11.767969 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:13:11.767980 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:11.768009 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:13:11.768019 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:11.768118 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:11.768192 (+ 74us) server_negotiation.cc:299] Negotiation successful
0620 20:13:11.768229 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 20:13:11.768237 27420 leader_election.cc:302] T 00000000000000000000000000000000 P 59e67743e1fc4cb9a983af828cbdc40c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59e67743e1fc4cb9a983af828cbdc40c; no voters:
I20260620 20:13:11.768801 27420 leader_election.cc:288] T 00000000000000000000000000000000 P 59e67743e1fc4cb9a983af828cbdc40c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:13:11.768882 27430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59e67743e1fc4cb9a983af828cbdc40c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:13:11.769073 27430 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59e67743e1fc4cb9a983af828cbdc40c [term 1 LEADER]: Becoming Leader. State: Replica: 59e67743e1fc4cb9a983af828cbdc40c, State: Running, Role: LEADER
I20260620 20:13:11.769675 27430 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59e67743e1fc4cb9a983af828cbdc40c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e67743e1fc4cb9a983af828cbdc40c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41929 } }
I20260620 20:13:11.770254 27420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59e67743e1fc4cb9a983af828cbdc40c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:13:11.770998 27430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59e67743e1fc4cb9a983af828cbdc40c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59e67743e1fc4cb9a983af828cbdc40c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e67743e1fc4cb9a983af828cbdc40c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41929 } } }
I20260620 20:13:11.771212 27430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59e67743e1fc4cb9a983af828cbdc40c [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:11.771473 27431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59e67743e1fc4cb9a983af828cbdc40c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59e67743e1fc4cb9a983af828cbdc40c. Latest consensus state: current_term: 1 leader_uuid: "59e67743e1fc4cb9a983af828cbdc40c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e67743e1fc4cb9a983af828cbdc40c" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 41929 } } }
I20260620 20:13:11.771613 27431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59e67743e1fc4cb9a983af828cbdc40c [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:11.771637 27439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:13:11.774853 27439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:13:11.781394 27439 catalog_manager.cc:1347] Generated new cluster ID: 9ff6b03ed716425481c26cce9bb153a5
I20260620 20:13:11.781461 27439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:13:11.794287 27439 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:13:11.794598 27439 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:13:11.802356 27439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59e67743e1fc4cb9a983af828cbdc40c: Generated new TSK 0
I20260620 20:13:11.803205 27439 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:13:11.810626 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 27352
2026-06-20T20:13:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1329 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:13:11 dist-test-slave-l8h4 krb5kdc[27451](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:13:11 dist-test-slave-l8h4 krb5kdc[27451](info): set up 2 sockets
Jun 20 20:13:11 dist-test-slave-l8h4 krb5kdc[27451](info): commencing operation
krb5kdc: starting...
W20260620 20:13:12.786971 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.958s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:13:12 dist-test-slave-l8h4 krb5kdc[27451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:13:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:13:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:13:12.913062 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:42361
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:38687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:42361
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:13:13.028100 27464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:13:13.028391 27464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:13:13.028451 27464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:13:13.032958 27464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:13:13.033057 27464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:13:13.033087 27464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:13:13.033108 27464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:13:13.033129 27464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:13:13.039145 27464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:38687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:42361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:42361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:13:13.040203 27464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:13:13.041195 27464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:13:13.050480 27470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:13.050437 27469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:13.050488 27472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:13.050729 27464 server_base.cc:1034] running on GCE node
I20260620 20:13:13.051299 27464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:13:13.052351 27464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:13:13.053535 27464 hybrid_clock.cc:648] HybridClock initialized: now 1781986393053525 us; error 28 us; skew 500 ppm
Jun 20 20:13:13 dist-test-slave-l8h4 krb5kdc[27451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986393, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:13:13.059599 27464 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:13:13.061007 27464 webserver.cc:460] Webserver started at http://127.5.91.190:34435/ using document root <none> and password file <none>
I20260620 20:13:13.061704 27464 fs_manager.cc:359] Metadata directory not provided
I20260620 20:13:13.061868 27464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:13:13.062248 27464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:13:13.064419 27464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "1e18741df2a04c29aaa918e24b88d536"
format_stamp: "Formatted at 2026-06-20 20:13:13 on dist-test-slave-l8h4"
server_key: "c1752b07732dd26f7f536b846bd216de"
server_key_iv: "6215a21272c60424d51f5901c7e72c7f"
server_key_version: "encryptionkey@0"
I20260620 20:13:13.064965 27464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "1e18741df2a04c29aaa918e24b88d536"
format_stamp: "Formatted at 2026-06-20 20:13:13 on dist-test-slave-l8h4"
server_key: "c1752b07732dd26f7f536b846bd216de"
server_key_iv: "6215a21272c60424d51f5901c7e72c7f"
server_key_version: "encryptionkey@0"
I20260620 20:13:13.068679 27464 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260620 20:13:13.071056 27479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:13:13.072284 27464 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:13:13.072398 27464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "1e18741df2a04c29aaa918e24b88d536"
format_stamp: "Formatted at 2026-06-20 20:13:13 on dist-test-slave-l8h4"
server_key: "c1752b07732dd26f7f536b846bd216de"
server_key_iv: "6215a21272c60424d51f5901c7e72c7f"
server_key_version: "encryptionkey@0"
I20260620 20:13:13.072490 27464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:13:13.097239 27464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:13:13.100446 27464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:13:13.100664 27464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:13:13.107471 27464 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:42361
I20260620 20:13:13.107512 27531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:42361 every 8 connection(s)
I20260620 20:13:13.108392 27464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:13:13.110282 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 27464
I20260620 20:13:13.110415 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:13:13.110720 5486 external_mini_cluster.cc:1428] Setting key eb5f012d5907f845557941ae41f83cf4
I20260620 20:13:13.111595 27532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:13:13.117995 27532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e18741df2a04c29aaa918e24b88d536: Bootstrap starting.
Jun 20 20:13:13 dist-test-slave-l8h4 krb5kdc[27451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:13:13.120735 27532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e18741df2a04c29aaa918e24b88d536: Neither blocks nor log segments found. Creating new log.
I20260620 20:13:13.121517 27532 log.cc:824] T 00000000000000000000000000000000 P 1e18741df2a04c29aaa918e24b88d536: Log is configured to *not* fsync() on all Append() calls
I20260620 20:13:13.123677 27532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e18741df2a04c29aaa918e24b88d536: No bootstrap required, opened a new log
I20260620 20:13:13.125514 27535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:13.112098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42471 (local address 127.5.91.190:42361)
0620 20:13:13.112491 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:13.112506 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:13.112540 (+ 34us) server_negotiation.cc:407] Connection header received
0620 20:13:13.113167 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:13.113182 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:13.113493 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:13.113770 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:13.114767 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:13.115756 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:13.116896 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:13.117363 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:13.117515 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:13.120325 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:13.120331 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:13.120343 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:13.120371 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:13.122216 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:13.122824 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:13.122830 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:13.122834 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:13.122951 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:13.123243 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:13.123245 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:13.123246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:13.124127 (+ 881us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:13.124281 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:13.124552 (+ 271us) server_negotiation.cc:299] Negotiation successful
0620 20:13:13.124844 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":123,"threads_started":1}
I20260620 20:13:13.127290 27532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e18741df2a04c29aaa918e24b88d536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e18741df2a04c29aaa918e24b88d536" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42361 } }
I20260620 20:13:13.127559 27532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e18741df2a04c29aaa918e24b88d536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:13:13.127833 27532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e18741df2a04c29aaa918e24b88d536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e18741df2a04c29aaa918e24b88d536, State: Initialized, Role: FOLLOWER
I20260620 20:13:13.128340 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:13:13.128459 27532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e18741df2a04c29aaa918e24b88d536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e18741df2a04c29aaa918e24b88d536" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42361 } }
I20260620 20:13:13.128643 27532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e18741df2a04c29aaa918e24b88d536 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:13:13.128708 27532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e18741df2a04c29aaa918e24b88d536 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:13:13.128801 27532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e18741df2a04c29aaa918e24b88d536 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:13:13.129700 27532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e18741df2a04c29aaa918e24b88d536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e18741df2a04c29aaa918e24b88d536" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42361 } }
I20260620 20:13:13.130112 27532 leader_election.cc:302] T 00000000000000000000000000000000 P 1e18741df2a04c29aaa918e24b88d536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e18741df2a04c29aaa918e24b88d536; no voters:
I20260620 20:13:13.130524 27532 leader_election.cc:288] T 00000000000000000000000000000000 P 1e18741df2a04c29aaa918e24b88d536 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:13:13.130574 27542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e18741df2a04c29aaa918e24b88d536 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:13:13.130779 27542 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e18741df2a04c29aaa918e24b88d536 [term 1 LEADER]: Becoming Leader. State: Replica: 1e18741df2a04c29aaa918e24b88d536, State: Running, Role: LEADER
I20260620 20:13:13.131081 27542 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e18741df2a04c29aaa918e24b88d536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e18741df2a04c29aaa918e24b88d536" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42361 } }
I20260620 20:13:13.131819 27532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e18741df2a04c29aaa918e24b88d536 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:13:13.132069 27542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e18741df2a04c29aaa918e24b88d536 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e18741df2a04c29aaa918e24b88d536. Latest consensus state: current_term: 1 leader_uuid: "1e18741df2a04c29aaa918e24b88d536" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e18741df2a04c29aaa918e24b88d536" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42361 } } }
I20260620 20:13:13.132180 27542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e18741df2a04c29aaa918e24b88d536 [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:13.132511 27546 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:13:13.133073 27543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e18741df2a04c29aaa918e24b88d536 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e18741df2a04c29aaa918e24b88d536" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e18741df2a04c29aaa918e24b88d536" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42361 } } }
I20260620 20:13:13.133208 27543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e18741df2a04c29aaa918e24b88d536 [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:13.136581 27546 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:13:13.139305 27535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:13.130313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45491 (local address 127.5.91.190:42361)
0620 20:13:13.130447 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:13.130451 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:13.130677 (+ 226us) server_negotiation.cc:407] Connection header received
0620 20:13:13.130855 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:13.130858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:13.130901 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:13.130945 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:13.131788 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:13.132564 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:13.133738 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:13.134061 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:13.134166 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:13.135601 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:13.135605 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:13.135608 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:13.135634 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:13.137267 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:13.137743 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:13.137747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:13.137749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:13.137851 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:13.138267 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:13.138270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:13.138272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:13.138485 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:13.138618 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:13.138895 (+ 277us) server_negotiation.cc:299] Negotiation successful
0620 20:13:13.139014 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 20:13:13.143304 27546 catalog_manager.cc:1347] Generated new cluster ID: 8ded437b93364492b5c154db15b18447
I20260620 20:13:13.143405 27546 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:13:13.154482 27546 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:13:13.155560 27546 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:13:13.164129 27546 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e18741df2a04c29aaa918e24b88d536: Generated new TSK 0
I20260620 20:13:13.165223 27546 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:13:13.175736 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 27464
2026-06-20T20:13:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1365 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:13:13 dist-test-slave-l8h4 krb5kdc[27563](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:13:13 dist-test-slave-l8h4 krb5kdc[27563](info): set up 2 sockets
Jun 20 20:13:13 dist-test-slave-l8h4 krb5kdc[27563](info): commencing operation
krb5kdc: starting...
W20260620 20:13:14.151011 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.956s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:13:14 dist-test-slave-l8h4 krb5kdc[27563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:13:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:13:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:13:14.270462 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:42285
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:42285
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:13:14.378604 27576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:13:14.378863 27576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:13:14.378902 27576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:13:14.382817 27576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:13:14.382872 27576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:13:14.382890 27576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:13:14.382906 27576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:13:14.382920 27576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:13:14.388036 27576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:42285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:42285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:13:14.389014 27576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:13:14.389952 27576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:13:14.397898 27584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:14.397926 27582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:14.398108 27581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:14.398274 27576 server_base.cc:1034] running on GCE node
I20260620 20:13:14.398835 27576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:13:14.399893 27576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:13:14.401062 27576 hybrid_clock.cc:648] HybridClock initialized: now 1781986394401044 us; error 25 us; skew 500 ppm
Jun 20 20:13:14 dist-test-slave-l8h4 krb5kdc[27563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986394, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:13:14.403664 27576 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:13:14.404488 27576 webserver.cc:460] Webserver started at http://127.5.91.190:34991/ using document root <none> and password file <none>
I20260620 20:13:14.404997 27576 fs_manager.cc:359] Metadata directory not provided
I20260620 20:13:14.405061 27576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:13:14.405213 27576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:13:14.407131 27576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "09f155b7126e44ebbd0935e7114c8dcf"
format_stamp: "Formatted at 2026-06-20 20:13:14 on dist-test-slave-l8h4"
server_key: "59b19dc5127b0c4ab8bb48f47ac22c79"
server_key_iv: "f5b54206cc0f0fc5cbc3e0e52b97a24f"
server_key_version: "encryptionkey@0"
I20260620 20:13:14.407609 27576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "09f155b7126e44ebbd0935e7114c8dcf"
format_stamp: "Formatted at 2026-06-20 20:13:14 on dist-test-slave-l8h4"
server_key: "59b19dc5127b0c4ab8bb48f47ac22c79"
server_key_iv: "f5b54206cc0f0fc5cbc3e0e52b97a24f"
server_key_version: "encryptionkey@0"
I20260620 20:13:14.411037 27576 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 20:13:14.413389 27591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:13:14.414743 27576 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 20:13:14.414856 27576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "09f155b7126e44ebbd0935e7114c8dcf"
format_stamp: "Formatted at 2026-06-20 20:13:14 on dist-test-slave-l8h4"
server_key: "59b19dc5127b0c4ab8bb48f47ac22c79"
server_key_iv: "f5b54206cc0f0fc5cbc3e0e52b97a24f"
server_key_version: "encryptionkey@0"
I20260620 20:13:14.414978 27576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:13:14.436139 27576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:13:14.439399 27576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:13:14.439587 27576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:13:14.446116 27576 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:42285
I20260620 20:13:14.446188 27643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:42285 every 8 connection(s)
I20260620 20:13:14.447055 27576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:13:14.449757 27644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:13:14.455689 27644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09f155b7126e44ebbd0935e7114c8dcf: Bootstrap starting.
I20260620 20:13:14.456486 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 27576
I20260620 20:13:14.456604 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:13:14.456844 5486 external_mini_cluster.cc:1428] Setting key 739bb7ef38512660929162de50e80653
I20260620 20:13:14.458418 27644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09f155b7126e44ebbd0935e7114c8dcf: Neither blocks nor log segments found. Creating new log.
I20260620 20:13:14.459343 27644 log.cc:824] T 00000000000000000000000000000000 P 09f155b7126e44ebbd0935e7114c8dcf: Log is configured to *not* fsync() on all Append() calls
I20260620 20:13:14.462096 27644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09f155b7126e44ebbd0935e7114c8dcf: No bootstrap required, opened a new log
I20260620 20:13:14.466250 27644 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09f155b7126e44ebbd0935e7114c8dcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f155b7126e44ebbd0935e7114c8dcf" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42285 } }
I20260620 20:13:14.466534 27644 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09f155b7126e44ebbd0935e7114c8dcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:13:14.466589 27644 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09f155b7126e44ebbd0935e7114c8dcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09f155b7126e44ebbd0935e7114c8dcf, State: Initialized, Role: FOLLOWER
I20260620 20:13:14.466917 27647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:14.458041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36533 (local address 127.5.91.190:42285)
0620 20:13:14.458493 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:14.458513 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:14.458556 (+ 43us) server_negotiation.cc:407] Connection header received
0620 20:13:14.459452 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:14.459476 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:14.459895 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:14.460211 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:14.460793 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:14.462256 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:14.463164 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:14.463858 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:14.464024 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:14.464824 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:14.464832 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:14.464848 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:13:14.465351 (+ 503us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:14.465419 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:13:14.465448 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:14.465545 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:14.465676 (+ 131us) server_negotiation.cc:299] Negotiation successful
0620 20:13:14.465883 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":103,"threads_started":1}
I20260620 20:13:14.467054 27644 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09f155b7126e44ebbd0935e7114c8dcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f155b7126e44ebbd0935e7114c8dcf" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42285 } }
I20260620 20:13:14.467201 27644 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09f155b7126e44ebbd0935e7114c8dcf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:13:14.467262 27644 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09f155b7126e44ebbd0935e7114c8dcf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:13:14.467353 27644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09f155b7126e44ebbd0935e7114c8dcf [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:13:14.468348 27644 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09f155b7126e44ebbd0935e7114c8dcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f155b7126e44ebbd0935e7114c8dcf" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42285 } }
I20260620 20:13:14.468695 27644 leader_election.cc:302] T 00000000000000000000000000000000 P 09f155b7126e44ebbd0935e7114c8dcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09f155b7126e44ebbd0935e7114c8dcf; no voters:
I20260620 20:13:14.469007 27644 leader_election.cc:288] T 00000000000000000000000000000000 P 09f155b7126e44ebbd0935e7114c8dcf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:13:14.469112 27649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09f155b7126e44ebbd0935e7114c8dcf [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:13:14.469329 27649 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09f155b7126e44ebbd0935e7114c8dcf [term 1 LEADER]: Becoming Leader. State: Replica: 09f155b7126e44ebbd0935e7114c8dcf, State: Running, Role: LEADER
I20260620 20:13:14.469661 27649 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09f155b7126e44ebbd0935e7114c8dcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f155b7126e44ebbd0935e7114c8dcf" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42285 } }
I20260620 20:13:14.469837 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:13:14.470244 27644 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09f155b7126e44ebbd0935e7114c8dcf [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:13:14.471131 27651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09f155b7126e44ebbd0935e7114c8dcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09f155b7126e44ebbd0935e7114c8dcf. Latest consensus state: current_term: 1 leader_uuid: "09f155b7126e44ebbd0935e7114c8dcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f155b7126e44ebbd0935e7114c8dcf" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42285 } } }
I20260620 20:13:14.471258 27651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09f155b7126e44ebbd0935e7114c8dcf [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:14.471542 27650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09f155b7126e44ebbd0935e7114c8dcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09f155b7126e44ebbd0935e7114c8dcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f155b7126e44ebbd0935e7114c8dcf" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42285 } } }
I20260620 20:13:14.471618 27650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09f155b7126e44ebbd0935e7114c8dcf [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:14.472375 27662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:13:14.475616 27662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:13:14.476199 27647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:14.472070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41595 (local address 127.5.91.190:42285)
0620 20:13:14.472187 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:14.472191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:14.472639 (+ 448us) server_negotiation.cc:407] Connection header received
0620 20:13:14.472859 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:14.472862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:14.472913 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:14.472997 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:14.473269 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:14.473769 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:14.474754 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:14.474978 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:14.475064 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:14.475817 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:14.475821 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:14.475824 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:13:14.475837 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:14.475862 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:13:14.475872 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:14.475928 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:14.476016 (+ 88us) server_negotiation.cc:299] Negotiation successful
0620 20:13:14.476063 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 20:13:14.481696 27662 catalog_manager.cc:1347] Generated new cluster ID: b945db8928494c34892867c7c799f3d2
I20260620 20:13:14.481824 27662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:13:14.498687 27662 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:13:14.499962 27662 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:13:14.515384 27662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09f155b7126e44ebbd0935e7114c8dcf: Generated new TSK 0
I20260620 20:13:14.516119 27662 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:13:14.527773 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 27576
2026-06-20T20:13:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1351 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:13:14 dist-test-slave-l8h4 krb5kdc[27675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:13:14 dist-test-slave-l8h4 krb5kdc[27675](info): set up 2 sockets
Jun 20 20:13:14 dist-test-slave-l8h4 krb5kdc[27675](info): commencing operation
krb5kdc: starting...
W20260620 20:13:15.498998 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.952s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:13:15 dist-test-slave-l8h4 krb5kdc[27675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:13:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:13:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:13:15.624624 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:42091
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:35535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:42091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:13:15.740780 27688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:13:15.741218 27688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:13:15.741271 27688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:13:15.747545 27688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:13:15.747654 27688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:13:15.747676 27688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:13:15.747695 27688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:13:15.747711 27688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:13:15.754432 27688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:35535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:42091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:42091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:13:15.755642 27688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:13:15.756631 27688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:13:15.765297 27694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:15.765568 27696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:15.765297 27693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:15.766683 27688 server_base.cc:1034] running on GCE node
I20260620 20:13:15.767405 27688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:13:15.768538 27688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:13:15.769726 27688 hybrid_clock.cc:648] HybridClock initialized: now 1781986395769705 us; error 38 us; skew 500 ppm
Jun 20 20:13:15 dist-test-slave-l8h4 krb5kdc[27675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986395, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:13:15.772557 27688 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:13:15.773428 27688 webserver.cc:460] Webserver started at http://127.5.91.190:33881/ using document root <none> and password file <none>
I20260620 20:13:15.773957 27688 fs_manager.cc:359] Metadata directory not provided
I20260620 20:13:15.774012 27688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:13:15.774237 27688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:13:15.777231 27688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "397200585b5643f6b4a0a3c046af5127"
format_stamp: "Formatted at 2026-06-20 20:13:15 on dist-test-slave-l8h4"
server_key: "f2dc5b844284cf6d6eb2c658db881583"
server_key_iv: "7cb68f37e19d4d9fdf3b589275ad6884"
server_key_version: "encryptionkey@0"
I20260620 20:13:15.777899 27688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "397200585b5643f6b4a0a3c046af5127"
format_stamp: "Formatted at 2026-06-20 20:13:15 on dist-test-slave-l8h4"
server_key: "f2dc5b844284cf6d6eb2c658db881583"
server_key_iv: "7cb68f37e19d4d9fdf3b589275ad6884"
server_key_version: "encryptionkey@0"
I20260620 20:13:15.782308 27688 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 20:13:15.784660 27703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:13:15.786244 27688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:13:15.786362 27688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "397200585b5643f6b4a0a3c046af5127"
format_stamp: "Formatted at 2026-06-20 20:13:15 on dist-test-slave-l8h4"
server_key: "f2dc5b844284cf6d6eb2c658db881583"
server_key_iv: "7cb68f37e19d4d9fdf3b589275ad6884"
server_key_version: "encryptionkey@0"
I20260620 20:13:15.786478 27688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:13:15.791230 27688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:13:15.794471 27688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:13:15.794638 27688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:13:15.801834 27755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:42091 every 8 connection(s)
I20260620 20:13:15.801849 27688 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:42091
I20260620 20:13:15.803170 27688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:13:15.806712 27756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:13:15.811278 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 27688
I20260620 20:13:15.811419 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:13:15.811653 5486 external_mini_cluster.cc:1428] Setting key d8f671ae68aee5474498ec72f1a23fa9
I20260620 20:13:15.813153 27756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 397200585b5643f6b4a0a3c046af5127: Bootstrap starting.
I20260620 20:13:15.815546 27756 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 397200585b5643f6b4a0a3c046af5127: Neither blocks nor log segments found. Creating new log.
I20260620 20:13:15.816336 27756 log.cc:824] T 00000000000000000000000000000000 P 397200585b5643f6b4a0a3c046af5127: Log is configured to *not* fsync() on all Append() calls
I20260620 20:13:15.817407 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:15.813142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45889 (local address 127.5.91.190:42091)
0620 20:13:15.814004 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:15.814027 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:15.814072 (+ 45us) server_negotiation.cc:407] Connection header received
0620 20:13:15.814788 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:15.814804 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:15.815096 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:15.815403 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:15.816134 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:15.816138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:15.816149 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:13:15.816160 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:15.816210 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:13:15.816229 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:15.816320 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:15.816422 (+ 102us) server_negotiation.cc:299] Negotiation successful
0620 20:13:15.816578 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":149,"threads_started":1}
I20260620 20:13:15.818382 27756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 397200585b5643f6b4a0a3c046af5127: No bootstrap required, opened a new log
I20260620 20:13:15.820267 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:13:15.822956 27756 raft_consensus.cc:348] T 00000000000000000000000000000000 P 397200585b5643f6b4a0a3c046af5127 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397200585b5643f6b4a0a3c046af5127" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42091 } }
I20260620 20:13:15.823266 27756 raft_consensus.cc:374] T 00000000000000000000000000000000 P 397200585b5643f6b4a0a3c046af5127 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 20:13:15.823325 27756 raft_consensus.cc:729] T 00000000000000000000000000000000 P 397200585b5643f6b4a0a3c046af5127 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 397200585b5643f6b4a0a3c046af5127, State: Initialized, Role: FOLLOWER
I20260620 20:13:15.823791 27756 consensus_queue.cc:260] T 00000000000000000000000000000000 P 397200585b5643f6b4a0a3c046af5127 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397200585b5643f6b4a0a3c046af5127" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42091 } }
I20260620 20:13:15.823942 27756 raft_consensus.cc:388] T 00000000000000000000000000000000 P 397200585b5643f6b4a0a3c046af5127 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 20:13:15.824007 27756 raft_consensus.cc:482] T 00000000000000000000000000000000 P 397200585b5643f6b4a0a3c046af5127 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 20:13:15.824115 27756 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 397200585b5643f6b4a0a3c046af5127 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:13:15.824249 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:15.822615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59655 (local address 127.5.91.190:42091)
0620 20:13:15.822731 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:15.822735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:15.822852 (+ 117us) server_negotiation.cc:407] Connection header received
0620 20:13:15.822961 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:15.822963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:15.822993 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:15.823058 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:15.823838 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:15.823841 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:15.823843 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 20:13:15.823853 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:15.823878 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 20:13:15.823889 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:15.823982 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:15.824091 (+ 109us) server_negotiation.cc:299] Negotiation successful
0620 20:13:15.824129 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 20:13:15.825325 27756 raft_consensus.cc:504] T 00000000000000000000000000000000 P 397200585b5643f6b4a0a3c046af5127 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397200585b5643f6b4a0a3c046af5127" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42091 } }
I20260620 20:13:15.825657 27756 leader_election.cc:302] T 00000000000000000000000000000000 P 397200585b5643f6b4a0a3c046af5127 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 397200585b5643f6b4a0a3c046af5127; no voters:
I20260620 20:13:15.825930 27756 leader_election.cc:288] T 00000000000000000000000000000000 P 397200585b5643f6b4a0a3c046af5127 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 20:13:15.826025 27766 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 397200585b5643f6b4a0a3c046af5127 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:13:15.826244 27766 raft_consensus.cc:686] T 00000000000000000000000000000000 P 397200585b5643f6b4a0a3c046af5127 [term 1 LEADER]: Becoming Leader. State: Replica: 397200585b5643f6b4a0a3c046af5127, State: Running, Role: LEADER
I20260620 20:13:15.826517 27766 consensus_queue.cc:237] T 00000000000000000000000000000000 P 397200585b5643f6b4a0a3c046af5127 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397200585b5643f6b4a0a3c046af5127" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42091 } }
I20260620 20:13:15.827106 27756 sys_catalog.cc:564] T 00000000000000000000000000000000 P 397200585b5643f6b4a0a3c046af5127 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:13:15.827965 27767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 397200585b5643f6b4a0a3c046af5127 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "397200585b5643f6b4a0a3c046af5127" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397200585b5643f6b4a0a3c046af5127" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42091 } } }
I20260620 20:13:15.828083 27767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 397200585b5643f6b4a0a3c046af5127 [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:15.828352 27768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 397200585b5643f6b4a0a3c046af5127 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 397200585b5643f6b4a0a3c046af5127. Latest consensus state: current_term: 1 leader_uuid: "397200585b5643f6b4a0a3c046af5127" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397200585b5643f6b4a0a3c046af5127" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 42091 } } }
I20260620 20:13:15.828441 27768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 397200585b5643f6b4a0a3c046af5127 [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:15.828687 27775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:13:15.831800 27775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:13:15.837025 27775 catalog_manager.cc:1347] Generated new cluster ID: 9d67165e55d544ac9e6b9a06cb16f4c6
I20260620 20:13:15.837106 27775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:13:15.845207 27775 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:13:15.845651 27775 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:13:15.851570 27775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 397200585b5643f6b4a0a3c046af5127: Generated new TSK 0
I20260620 20:13:15.852308 27775 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 20:13:15.859431 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 27688
2026-06-20T20:13:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1333 ms)
[----------] 18 tests from AuthTokenIssuingTest (24398 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:13:15 dist-test-slave-l8h4 krb5kdc[27787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:13:15 dist-test-slave-l8h4 krb5kdc[27787](info): set up 2 sockets
Jun 20 20:13:15 dist-test-slave-l8h4 krb5kdc[27787](info): commencing operation
krb5kdc: starting...
W20260620 20:13:16.847127 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.966s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:13:16 dist-test-slave-l8h4 krb5kdc[27787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:13:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:13:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:13:16.965540 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:44805
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:37601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:44805,127.5.91.189:45661,127.5.91.188:39693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:13:17.077898 27800 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 20:13:17.078245 27800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:13:17.078292 27800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:13:17.078352 27800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:13:17.083190 27800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:13:17.083257 27800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:13:17.083276 27800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:13:17.083292 27800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:13:17.083308 27800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:13:17.088892 27800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:37601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:44805,127.5.91.189:45661,127.5.91.188:39693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:44805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:13:17.089941 27800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:13:17.090953 27800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:13:17.099452 27806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:17.099469 27805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:17.099454 27808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:17.099634 27800 server_base.cc:1034] running on GCE node
I20260620 20:13:17.100338 27800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:13:17.101483 27800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:13:17.102648 27800 hybrid_clock.cc:648] HybridClock initialized: now 1781986397102634 us; error 29 us; skew 500 ppm
Jun 20 20:13:17 dist-test-slave-l8h4 krb5kdc[27787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986397, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:13:17.105317 27800 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:13:17.106225 27800 webserver.cc:460] Webserver started at http://127.5.91.190:33107/ using document root <none> and password file <none>
I20260620 20:13:17.106745 27800 fs_manager.cc:359] Metadata directory not provided
I20260620 20:13:17.106809 27800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:13:17.106992 27800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:13:17.108954 27800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997"
format_stamp: "Formatted at 2026-06-20 20:13:17 on dist-test-slave-l8h4"
server_key: "5d7d4efe3fa9c6a4c4d81f567ebda09d"
server_key_iv: "fe87b1235a848301771b462ef223b429"
server_key_version: "encryptionkey@0"
I20260620 20:13:17.109483 27800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997"
format_stamp: "Formatted at 2026-06-20 20:13:17 on dist-test-slave-l8h4"
server_key: "5d7d4efe3fa9c6a4c4d81f567ebda09d"
server_key_iv: "fe87b1235a848301771b462ef223b429"
server_key_version: "encryptionkey@0"
I20260620 20:13:17.113226 27800 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 20:13:17.115728 27815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:13:17.117038 27800 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 20:13:17.117166 27800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997"
format_stamp: "Formatted at 2026-06-20 20:13:17 on dist-test-slave-l8h4"
server_key: "5d7d4efe3fa9c6a4c4d81f567ebda09d"
server_key_iv: "fe87b1235a848301771b462ef223b429"
server_key_version: "encryptionkey@0"
I20260620 20:13:17.117264 27800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:13:17.129092 27800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:13:17.132086 27800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:13:17.132262 27800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:13:17.138827 27800 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:44805
I20260620 20:13:17.139119 27867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:44805 every 8 connection(s)
I20260620 20:13:17.139727 27800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:13:17.141594 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 27800
I20260620 20:13:17.141702 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:13:17.141935 5486 external_mini_cluster.cc:1428] Setting key 775764d41583ec8eeef2357c54978ab7
I20260620 20:13:17.142683 27868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:13:17.147401 27868 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44805 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.91.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.189@KRBTEST.COM" created.
Jun 20 20:13:17 dist-test-slave-l8h4 krb5kdc[27787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986397, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:13:17.163998 27872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.150090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39030 (local address 127.5.91.190:44805)
0620 20:13:17.150644 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.150660 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.150807 (+ 147us) server_negotiation.cc:407] Connection header received
0620 20:13:17.151812 (+ 1005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.151816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.151989 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.152145 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.152916 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.154044 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.155104 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.155455 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.155607 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.158351 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:17.158359 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:17.158375 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:17.158406 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:17.160618 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.161242 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.161247 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.161249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.161364 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.161777 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.161780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.161781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.162255 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:17.162366 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:17.162860 (+ 494us) server_negotiation.cc:299] Negotiation successful
0620 20:13:17.163085 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"spinlock_wait_cycles":57600,"thread_start_us":152,"threads_started":1}
I20260620 20:13:17.164003 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.150090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:44805 (local address 127.0.0.1:39030)
0620 20:13:17.150664 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.150713 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 20:13:17.151535 (+ 822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:13:17.152155 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:13:17.152160 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:13:17.152560 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:13:17.152785 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:17.152798 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.154263 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.154266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:17.154984 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:17.155003 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.155603 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.155607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:17.155709 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.156313 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:13:17.156332 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:13:17.158164 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:13:17.160772 (+ 2608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:17.160779 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:17.160790 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:17.161102 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:17.161459 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:17.161462 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:17.161464 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:17.161616 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:17.162439 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:13:17.162447 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:13:17.162595 (+ 148us) client_negotiation.cc:769] Sending connection context
0620 20:13:17.162756 (+ 161us) client_negotiation.cc:240] Negotiation successful
0620 20:13:17.163133 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":84,"thread_start_us":152,"threads_started":1}
I20260620 20:13:17.166347 27868 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } has no permanent_uuid. Determining permanent_uuid...
I20260620 20:13:17.167167 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.166869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:45661 (local address 127.0.0.1:46908)
0620 20:13:17.167008 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.167088 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.189:45661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260620 20:13:17.167351 27816 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.91.189:45661: connect: Connection refused (error 111)
I20260620 20:13:17.168179 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.167955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:45661 (local address 127.0.0.1:46916)
0620 20:13:17.168078 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.168127 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.189:45661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260620 20:13:17.168363 27868 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.189:45661: Network error: Client connection negotiation failed: client connection to 127.5.91.189:45661: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.189.keytab.
Entry for principal kudu/127.5.91.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.189.keytab.
I20260620 20:13:17.169656 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.189:45661
--webserver_interface=127.5.91.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.189.keytab
--principal=kudu/127.5.91.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:37601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:44805,127.5.91.189:45661,127.5.91.188:39693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 20:13:17.217628 27868 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } attempt: 1
I20260620 20:13:17.218564 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.218207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:45661 (local address 127.0.0.1:46920)
0620 20:13:17.218376 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.218460 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.189:45661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260620 20:13:17.219436 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.219164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:45661 (local address 127.0.0.1:46928)
0620 20:13:17.219300 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.219357 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.189:45661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260620 20:13:17.219653 27868 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.189:45661: Network error: Client connection negotiation failed: client connection to 127.5.91.189:45661: connect: Connection refused (error 111)
W20260620 20:13:17.283684 27875 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 20:13:17.284021 27875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:13:17.284073 27875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:13:17.284152 27875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260620 20:13:17.287884 27868 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } attempt: 2
W20260620 20:13:17.288498 27875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:13:17.288568 27875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:13:17.288595 27875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:13:17.288615 27875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:13:17.288645 27875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:13:17.288786 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.288436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:45661 (local address 127.0.0.1:46934)
0620 20:13:17.288622 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.288698 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.189:45661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260620 20:13:17.289645 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.289403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:45661 (local address 127.0.0.1:46946)
0620 20:13:17.289516 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.289577 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.189:45661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260620 20:13:17.289794 27868 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.189:45661: Network error: Client connection negotiation failed: client connection to 127.5.91.189:45661: connect: Connection refused (error 111)
I20260620 20:13:17.294723 27875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:37601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:44805,127.5.91.189:45661,127.5.91.188:39693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.189.keytab
--principal=kudu/127.5.91.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.189:45661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.91.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:13:17.296145 27875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:13:17.297159 27875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:13:17.305931 27883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:17.306018 27881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:17.306032 27880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:17.306329 27875 server_base.cc:1034] running on GCE node
I20260620 20:13:17.306993 27875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:13:17.308156 27875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:13:17.309399 27875 hybrid_clock.cc:648] HybridClock initialized: now 1781986397309314 us; error 48 us; skew 500 ppm
Jun 20 20:13:17 dist-test-slave-l8h4 krb5kdc[27787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986397, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:13:17.312304 27875 init.cc:377] Logged in from keytab as kudu/127.5.91.189@KRBTEST.COM (short username kudu)
I20260620 20:13:17.313251 27875 webserver.cc:460] Webserver started at http://127.5.91.189:43823/ using document root <none> and password file <none>
I20260620 20:13:17.313783 27875 fs_manager.cc:359] Metadata directory not provided
I20260620 20:13:17.313849 27875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:13:17.314045 27875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:13:17.316954 27875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-1/data/instance:
uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2"
format_stamp: "Formatted at 2026-06-20 20:13:17 on dist-test-slave-l8h4"
server_key: "5622e49ed5d2e8fbce57d8174a7961b5"
server_key_iv: "8a77eaf8a60a7806a4dc73dc0b9670e0"
server_key_version: "encryptionkey@0"
I20260620 20:13:17.317828 27875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-1/wal/instance:
uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2"
format_stamp: "Formatted at 2026-06-20 20:13:17 on dist-test-slave-l8h4"
server_key: "5622e49ed5d2e8fbce57d8174a7961b5"
server_key_iv: "8a77eaf8a60a7806a4dc73dc0b9670e0"
server_key_version: "encryptionkey@0"
I20260620 20:13:17.323362 27875 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260620 20:13:17.327301 27890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:13:17.328871 27875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260620 20:13:17.329020 27875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-1/wal
uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2"
format_stamp: "Formatted at 2026-06-20 20:13:17 on dist-test-slave-l8h4"
server_key: "5622e49ed5d2e8fbce57d8174a7961b5"
server_key_iv: "8a77eaf8a60a7806a4dc73dc0b9670e0"
server_key_version: "encryptionkey@0"
I20260620 20:13:17.329145 27875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:13:17.339668 27875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:13:17.344748 27875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:13:17.344959 27875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:13:17.352065 27875 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.189:45661
I20260620 20:13:17.352465 27942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.189:45661 every 8 connection(s)
I20260620 20:13:17.353092 27875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-1/data/info.pb
I20260620 20:13:17.356017 27943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:13:17.356211 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 27875
I20260620 20:13:17.356321 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-1/wal/instance
I20260620 20:13:17.356588 5486 external_mini_cluster.cc:1428] Setting key 7c08ceb4fff8c2d1e47df23d60534b9f
I20260620 20:13:17.360692 27943 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44805 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.91.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.188@KRBTEST.COM" created.
Jun 20 20:13:17 dist-test-slave-l8h4 krb5kdc[27787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986397, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.189@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:13:17.375625 27872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.362546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39034 (local address 127.5.91.190:44805)
0620 20:13:17.362703 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.362708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.363543 (+ 835us) server_negotiation.cc:407] Connection header received
0620 20:13:17.364449 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.364453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.364511 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.364627 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.365842 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.366505 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.367925 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.368171 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.368263 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.371482 (+ 3219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:17.371486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:17.371490 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:17.371521 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:17.373238 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.373851 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.373855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.373857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.373951 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.374356 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.374360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.374362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.374598 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:17.374816 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:17.375374 (+ 558us) server_negotiation.cc:299] Negotiation successful
0620 20:13:17.375485 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 20:13:17.376441 27946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.362915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:44805 (local address 127.0.0.1:39034)
0620 20:13:17.363385 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.363433 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 20:13:17.364254 (+ 821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:13:17.364825 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:13:17.364832 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:13:17.365188 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:13:17.365690 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:17.365704 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.366648 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.366652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:17.367785 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:17.367797 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.368324 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.368328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:17.368510 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.369222 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:13:17.369238 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:13:17.371230 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:13:17.373400 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:17.373406 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:17.373421 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:17.373717 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:17.374055 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:17.374058 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:17.374059 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:17.374263 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:17.374845 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:13:17.374850 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:13:17.375111 (+ 261us) client_negotiation.cc:769] Sending connection context
0620 20:13:17.375296 (+ 185us) client_negotiation.cc:240] Negotiation successful
0620 20:13:17.375561 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":135,"threads_started":1}
I20260620 20:13:17.377918 27943 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } has no permanent_uuid. Determining permanent_uuid...
I20260620 20:13:17.380954 27868 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } attempt: 3
Jun 20 20:13:17 dist-test-slave-l8h4 krb5kdc[27787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986397, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.189@KRBTEST.COM for kudu/127.5.91.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.188.keytab.
Entry for principal kudu/127.5.91.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.188.keytab.
I20260620 20:13:17.385680 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.188:39693
--webserver_interface=127.5.91.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.188.keytab
--principal=kudu/127.5.91.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:37601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:44805,127.5.91.189:45661,127.5.91.188:39693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 20:13:17 dist-test-slave-l8h4 krb5kdc[27787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986397, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for kudu/127.5.91.189@KRBTEST.COM
I20260620 20:13:17.389612 27946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.378385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:45661 (local address 127.0.0.1:46958)
0620 20:13:17.378499 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.378513 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:13:17.378585 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:13:17.379174 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:13:17.379176 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:13:17.379350 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:13:17.379440 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:17.379445 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.380431 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.380435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:17.381160 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:17.381170 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.381877 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.381880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:17.382062 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.382545 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:13:17.382556 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:13:17.383857 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:13:17.388115 (+ 4258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:17.388119 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:17.388121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:17.388339 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:17.388631 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:17.388634 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:17.388636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:17.388725 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:17.389141 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:13:17.389144 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:13:17.389233 (+ 89us) client_negotiation.cc:769] Sending connection context
0620 20:13:17.389327 (+ 94us) client_negotiation.cc:240] Negotiation successful
0620 20:13:17.389451 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260620 20:13:17.389766 27948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.378553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46958 (local address 127.5.91.189:45661)
0620 20:13:17.378829 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.378837 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.378854 (+ 17us) server_negotiation.cc:407] Connection header received
0620 20:13:17.378882 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.378886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.379026 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.379129 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.379531 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.380325 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.381269 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.381757 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.381836 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.384055 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:17.384075 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:17.384099 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:17.384142 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:17.387997 (+ 3855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.388448 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.388454 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.388456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.388543 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.388790 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.388793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.388795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.389048 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:17.389181 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:17.389428 (+ 247us) server_negotiation.cc:299] Negotiation successful
0620 20:13:17.389571 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":128,"threads_started":1}
I20260620 20:13:17.390690 27949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.381475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46968 (local address 127.5.91.189:45661)
0620 20:13:17.381736 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.381742 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.381752 (+ 10us) server_negotiation.cc:407] Connection header received
0620 20:13:17.381776 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.381778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.381809 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.381928 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.382384 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.382980 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.383984 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.384553 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.384679 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.386969 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:17.386975 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:17.386979 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:17.387007 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:17.388561 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.389061 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.389064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.389065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.389149 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.389459 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.389462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.389464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.389646 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:17.390077 (+ 431us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:17.390425 (+ 348us) server_negotiation.cc:299] Negotiation successful
0620 20:13:17.390534 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":88,"threads_started":1}
I20260620 20:13:17.390717 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.381364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:45661 (local address 127.0.0.1:46968)
0620 20:13:17.381474 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.381488 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:13:17.381550 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:13:17.381954 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:13:17.381956 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:13:17.382088 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:13:17.382243 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:17.382249 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.383084 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.383087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:17.383742 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:17.383753 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.384689 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.384693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:17.384795 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.385146 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:13:17.385160 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:13:17.386344 (+ 1184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:13:17.388726 (+ 2382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:17.388729 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:17.388731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:17.388967 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:17.389265 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:17.389268 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:17.389270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:17.389360 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:17.390092 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:13:17.390095 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:13:17.390233 (+ 138us) client_negotiation.cc:769] Sending connection context
0620 20:13:17.390366 (+ 133us) client_negotiation.cc:240] Negotiation successful
0620 20:13:17.390511 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260620 20:13:17.392136 27943 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } has no permanent_uuid. Determining permanent_uuid...
I20260620 20:13:17.392530 27868 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } has no permanent_uuid. Determining permanent_uuid...
I20260620 20:13:17.393054 27946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.392640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:39693 (local address 127.0.0.1:40748)
0620 20:13:17.392898 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.392982 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
W20260620 20:13:17.393234 27891 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.91.188:39693: connect: Connection refused (error 111)
I20260620 20:13:17.393353 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.393064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:39693 (local address 127.0.0.1:40758)
0620 20:13:17.393230 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.393281 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260620 20:13:17.394050 27946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.393848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:39693 (local address 127.0.0.1:40778)
0620 20:13:17.393936 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.393988 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260620 20:13:17.394116 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.393838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:39693 (local address 127.0.0.1:40770)
0620 20:13:17.394019 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.394060 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260620 20:13:17.394214 27943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.188:39693: Network error: Client connection negotiation failed: client connection to 127.5.91.188:39693: connect: Connection refused (error 111)
W20260620 20:13:17.394281 27868 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.188:39693: Network error: Client connection negotiation failed: client connection to 127.5.91.188:39693: connect: Connection refused (error 111)
I20260620 20:13:17.425467 27868 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } attempt: 1
I20260620 20:13:17.426425 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.426010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:39693 (local address 127.0.0.1:40792)
0620 20:13:17.426256 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.426329 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260620 20:13:17.427178 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.426930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:39693 (local address 127.0.0.1:40808)
0620 20:13:17.427059 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.427112 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260620 20:13:17.427347 27868 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.188:39693: Network error: Client connection negotiation failed: client connection to 127.5.91.188:39693: connect: Connection refused (error 111)
I20260620 20:13:17.443466 27943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } attempt: 1
I20260620 20:13:17.444639 27946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.444296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:39693 (local address 127.0.0.1:40814)
0620 20:13:17.444466 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.444545 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260620 20:13:17.445396 27946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.445184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:39693 (local address 127.0.0.1:40826)
0620 20:13:17.445285 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.445336 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 20:13:17.445528 27943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.188:39693: Network error: Client connection negotiation failed: client connection to 127.5.91.188:39693: connect: Connection refused (error 111)
W20260620 20:13:17.495242 27950 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 20:13:17.495513 27950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:13:17.495543 27950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:13:17.495589 27950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:13:17.499539 27950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:13:17.499600 27950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:13:17.499619 27950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:13:17.499634 27950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:13:17.499647 27950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:13:17.502498 27868 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } attempt: 2
I20260620 20:13:17.503291 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.502985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:39693 (local address 127.0.0.1:40836)
0620 20:13:17.503138 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.503202 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260620 20:13:17.504087 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.503888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:39693 (local address 127.0.0.1:40842)
0620 20:13:17.503981 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.504033 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260620 20:13:17.504200 27868 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.188:39693: Network error: Client connection negotiation failed: client connection to 127.5.91.188:39693: connect: Connection refused (error 111)
I20260620 20:13:17.505726 27950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:37601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:44805,127.5.91.189:45661,127.5.91.188:39693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.188.keytab
--principal=kudu/127.5.91.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.188:39693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.91.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:13:17.506847 27950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:13:17.507850 27950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 20:13:17.513729 27943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } attempt: 2
I20260620 20:13:17.514830 27946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.514255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:39693 (local address 127.0.0.1:40850)
0620 20:13:17.514513 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.514622 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260620 20:13:17.516247 27946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.515657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:39693 (local address 127.0.0.1:40866)
0620 20:13:17.515849 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.515927 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260620 20:13:17.516430 27957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:17.516484 27943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.188:39693: Network error: Client connection negotiation failed: client connection to 127.5.91.188:39693: connect: Connection refused (error 111)
W20260620 20:13:17.516463 27956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:17.516672 27959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:17.516784 27950 server_base.cc:1034] running on GCE node
I20260620 20:13:17.517323 27950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:13:17.518481 27950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:13:17.519670 27950 hybrid_clock.cc:648] HybridClock initialized: now 1781986397519640 us; error 36 us; skew 500 ppm
Jun 20 20:13:17 dist-test-slave-l8h4 krb5kdc[27787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986397, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:13:17.522590 27950 init.cc:377] Logged in from keytab as kudu/127.5.91.188@KRBTEST.COM (short username kudu)
I20260620 20:13:17.523447 27950 webserver.cc:460] Webserver started at http://127.5.91.188:45589/ using document root <none> and password file <none>
I20260620 20:13:17.523983 27950 fs_manager.cc:359] Metadata directory not provided
I20260620 20:13:17.524046 27950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:13:17.524235 27950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:13:17.526237 27950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-2/data/instance:
uuid: "4966ae8b2a9842a2a6b9d9be8b059df5"
format_stamp: "Formatted at 2026-06-20 20:13:17 on dist-test-slave-l8h4"
server_key: "4dd7b327936b372cc0080db5b8c37690"
server_key_iv: "743df7a478a9bb3d4a9f202f85d322db"
server_key_version: "encryptionkey@0"
I20260620 20:13:17.526772 27950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-2/wal/instance:
uuid: "4966ae8b2a9842a2a6b9d9be8b059df5"
format_stamp: "Formatted at 2026-06-20 20:13:17 on dist-test-slave-l8h4"
server_key: "4dd7b327936b372cc0080db5b8c37690"
server_key_iv: "743df7a478a9bb3d4a9f202f85d322db"
server_key_version: "encryptionkey@0"
I20260620 20:13:17.530375 27950 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:13:17.532930 27966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:13:17.534250 27950 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 20:13:17.534358 27950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-2/wal
uuid: "4966ae8b2a9842a2a6b9d9be8b059df5"
format_stamp: "Formatted at 2026-06-20 20:13:17 on dist-test-slave-l8h4"
server_key: "4dd7b327936b372cc0080db5b8c37690"
server_key_iv: "743df7a478a9bb3d4a9f202f85d322db"
server_key_version: "encryptionkey@0"
I20260620 20:13:17.534474 27950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:13:17.547704 27950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:13:17.550742 27950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:13:17.550910 27950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:13:17.557729 27950 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.188:39693
I20260620 20:13:17.557996 28018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.188:39693 every 8 connection(s)
I20260620 20:13:17.558733 27950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-2/data/info.pb
I20260620 20:13:17.561352 28019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:13:17.563099 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 27950
I20260620 20:13:17.563189 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781986111135528-5486-0/minicluster-data/master-2/wal/instance
I20260620 20:13:17.563446 5486 external_mini_cluster.cc:1428] Setting key 67fd990db9411d06ea22279f92e95cba
I20260620 20:13:17.565196 28019 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44805 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 20:13:17 dist-test-slave-l8h4 krb5kdc[27787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.188@KRBTEST.COM
Jun 20 20:13:17 dist-test-slave-l8h4 krb5kdc[27787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986397, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.188@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:13:17.577397 27872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.567740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39040 (local address 127.5.91.190:44805)
0620 20:13:17.567879 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.567884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.568195 (+ 311us) server_negotiation.cc:407] Connection header received
0620 20:13:17.568301 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.568304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.568350 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.568433 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.569042 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.569510 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.570493 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.570744 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.570869 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.573486 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:17.573490 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:17.573492 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:17.573520 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:17.575443 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.575983 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.575986 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.575987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.576070 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.576334 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.576337 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.576338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.576546 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:17.576631 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:17.577118 (+ 487us) server_negotiation.cc:299] Negotiation successful
0620 20:13:17.577238 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 20:13:17.578087 28022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.564715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40874 (local address 127.5.91.188:39693)
0620 20:13:17.565102 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.565117 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.565151 (+ 34us) server_negotiation.cc:407] Connection header received
0620 20:13:17.566232 (+ 1081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.566255 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.566609 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.566879 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.567732 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.568930 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.569828 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.570316 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.570429 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.572985 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:17.572990 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:17.572999 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:17.573021 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:17.574976 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.575481 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.575487 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.575493 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.575619 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.575911 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.575913 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.575914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.576421 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:17.576627 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:17.576859 (+ 232us) server_negotiation.cc:299] Negotiation successful
0620 20:13:17.577300 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":258,"thread_start_us":111,"threads_started":1}
I20260620 20:13:17.578091 28023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.567631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:44805 (local address 127.0.0.1:39040)
0620 20:13:17.568061 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.568115 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 20:13:17.568226 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:13:17.568476 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:13:17.568480 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:13:17.568777 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:13:17.568920 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:17.568926 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.569612 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.569615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:17.570357 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:17.570368 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.570908 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.570912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:17.571006 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.571488 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:13:17.571509 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:13:17.573342 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:13:17.575558 (+ 2216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:17.575566 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:17.575570 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:17.575870 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:17.576163 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:17.576166 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:17.576167 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:17.576250 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:17.576655 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:13:17.576662 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:13:17.576815 (+ 153us) client_negotiation.cc:769] Sending connection context
0620 20:13:17.577039 (+ 224us) client_negotiation.cc:240] Negotiation successful
0620 20:13:17.577224 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":2,"thread_start_us":166,"threads_started":1}
I20260620 20:13:17.579321 28019 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 20:13:17 dist-test-slave-l8h4 krb5kdc[27787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986397, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.188@KRBTEST.COM for kudu/127.5.91.189@KRBTEST.COM
Jun 20 20:13:17 dist-test-slave-l8h4 krb5kdc[27787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:13:17.588817 27949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.580123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46970 (local address 127.5.91.189:45661)
0620 20:13:17.580367 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.580371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.580389 (+ 18us) server_negotiation.cc:407] Connection header received
0620 20:13:17.580430 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.580432 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.580478 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.580549 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.580923 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.581493 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.582744 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.583164 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.583272 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.585414 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:17.585417 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:17.585419 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:17.585440 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:17.586932 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.587378 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.587381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.587383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.587500 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.587930 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.587935 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.587936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.588198 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:17.588284 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:17.588589 (+ 305us) server_negotiation.cc:299] Negotiation successful
0620 20:13:17.588691 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260620 20:13:17.588820 28023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.580071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:45661 (local address 127.0.0.1:46970)
0620 20:13:17.580220 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.580235 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:13:17.580308 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:13:17.580556 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:13:17.580558 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:13:17.580699 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:13:17.580781 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:17.580787 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.581885 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.581888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:17.582643 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:17.582651 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.583304 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.583308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:17.583397 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.583861 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:13:17.583874 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:13:17.585277 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:13:17.587027 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:17.587030 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:17.587032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:17.587298 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:17.587617 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:17.587621 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:17.587622 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:17.587715 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:17.588295 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:13:17.588299 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:13:17.588401 (+ 102us) client_negotiation.cc:769] Sending connection context
0620 20:13:17.588509 (+ 108us) client_negotiation.cc:240] Negotiation successful
0620 20:13:17.588648 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260620 20:13:17.589867 28019 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } has no permanent_uuid. Determining permanent_uuid...
I20260620 20:13:17.590723 27872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.580941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39048 (local address 127.5.91.190:44805)
0620 20:13:17.581628 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.581631 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.581640 (+ 9us) server_negotiation.cc:407] Connection header received
0620 20:13:17.581667 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.581669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.581700 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.581761 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.582508 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.583165 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.584135 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.584366 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.584462 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.587036 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:17.587040 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:17.587042 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:17.587068 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:17.588862 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.589281 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.589284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.589286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.589377 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.589622 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.589625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.589626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.589833 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:17.590026 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:17.590251 (+ 225us) server_negotiation.cc:299] Negotiation successful
0620 20:13:17.590366 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620}
Jun 20 20:13:17 dist-test-slave-l8h4 krb5kdc[27787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986397, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.188@KRBTEST.COM for kudu/127.5.91.188@KRBTEST.COM
Jun 20 20:13:17 dist-test-slave-l8h4 krb5kdc[27787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.189@KRBTEST.COM
I20260620 20:13:17.598007 28023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.590408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:39693 (local address 127.0.0.1:40886)
0620 20:13:17.590505 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.590517 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 20:13:17.590600 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:13:17.590813 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:13:17.590816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:13:17.590967 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:13:17.591076 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:17.591083 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.591857 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.591860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:17.592599 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:17.592610 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.593146 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.593149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:17.593205 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.593611 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:13:17.593637 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:13:17.595112 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:13:17.596626 (+ 1514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:17.596629 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:17.596631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:17.596795 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:17.597062 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:17.597065 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:17.597066 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:17.597131 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:17.597494 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:13:17.597497 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:13:17.597579 (+ 82us) client_negotiation.cc:769] Sending connection context
0620 20:13:17.597674 (+ 95us) client_negotiation.cc:240] Negotiation successful
0620 20:13:17.597810 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":1}
I20260620 20:13:17.598021 28022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.590442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40886 (local address 127.5.91.188:39693)
0620 20:13:17.590561 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.590565 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.590578 (+ 13us) server_negotiation.cc:407] Connection header received
0620 20:13:17.590677 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.590680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.590719 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.590792 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.591156 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.591751 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.592716 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.593003 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.593151 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.595230 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:17.595234 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:17.595236 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:17.595259 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:17.596526 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.596894 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.596897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.596899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.596980 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.597207 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.597210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.597212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.597393 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:17.597464 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:17.597732 (+ 268us) server_negotiation.cc:299] Negotiation successful
0620 20:13:17.597861 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":36}
I20260620 20:13:17.600270 27949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.591877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46986 (local address 127.5.91.189:45661)
0620 20:13:17.591971 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.591974 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.591987 (+ 13us) server_negotiation.cc:407] Connection header received
0620 20:13:17.592034 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.592070 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.592104 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.592162 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.592836 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.593471 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.594466 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.594810 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.594909 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.597284 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:17.597287 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:17.597289 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:17.597308 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:17.598777 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.599193 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.599197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.599198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.599281 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.599577 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.599579 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.599580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.599736 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:17.599865 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:17.600059 (+ 194us) server_negotiation.cc:299] Negotiation successful
0620 20:13:17.600147 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 20:13:17.602360 28019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4966ae8b2a9842a2a6b9d9be8b059df5: Bootstrap starting.
I20260620 20:13:17.603333 27868 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } attempt: 3
I20260620 20:13:17.603700 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:13:17.605505 28019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4966ae8b2a9842a2a6b9d9be8b059df5: Neither blocks nor log segments found. Creating new log.
I20260620 20:13:17.605059 27842 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39048
I20260620 20:13:17.606428 28019 log.cc:824] T 00000000000000000000000000000000 P 4966ae8b2a9842a2a6b9d9be8b059df5: Log is configured to *not* fsync() on all Append() calls
I20260620 20:13:17.607636 27943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } attempt: 3
Jun 20 20:13:17 dist-test-slave-l8h4 krb5kdc[27787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986397, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for kudu/127.5.91.188@KRBTEST.COM
I20260620 20:13:17.608855 28019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4966ae8b2a9842a2a6b9d9be8b059df5: No bootstrap required, opened a new log
I20260620 20:13:17.612151 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.603748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:39693 (local address 127.0.0.1:40902)
0620 20:13:17.603868 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.603882 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:13:17.603958 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:13:17.604395 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:13:17.604398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:13:17.604560 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:13:17.604655 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:17.604661 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.605402 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.605406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:17.605969 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:17.605978 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.606452 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.606455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:17.606534 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.607062 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:13:17.607077 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:13:17.608497 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:13:17.610563 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:17.610567 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:17.610569 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:17.610816 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:17.611074 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:17.611077 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:17.611079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:17.611167 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:17.611612 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:13:17.611615 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:13:17.611720 (+ 105us) client_negotiation.cc:769] Sending connection context
0620 20:13:17.611821 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 20:13:17.611962 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260620 20:13:17.612586 28022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.604088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40902 (local address 127.5.91.188:39693)
0620 20:13:17.604205 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.604208 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.604221 (+ 13us) server_negotiation.cc:407] Connection header received
0620 20:13:17.604250 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.604252 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.604285 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.604355 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.604768 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.605308 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.606071 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.606352 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.606536 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.608646 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:17.608650 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:17.608653 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:17.608677 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:17.610469 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.610908 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.610911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.610913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.610981 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.611257 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.611260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.611262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.611508 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:17.611601 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:17.612350 (+ 749us) server_negotiation.cc:299] Negotiation successful
0620 20:13:17.612456 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 20:13:17.612751 28019 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4966ae8b2a9842a2a6b9d9be8b059df5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44805 } } peers { permanent_uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } } peers { permanent_uuid: "4966ae8b2a9842a2a6b9d9be8b059df5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } }
I20260620 20:13:17.612995 28019 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4966ae8b2a9842a2a6b9d9be8b059df5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4966ae8b2a9842a2a6b9d9be8b059df5, State: Initialized, Role: FOLLOWER
I20260620 20:13:17.613299 28019 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4966ae8b2a9842a2a6b9d9be8b059df5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44805 } } peers { permanent_uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } } peers { permanent_uuid: "4966ae8b2a9842a2a6b9d9be8b059df5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } }
Jun 20 20:13:17 dist-test-slave-l8h4 krb5kdc[27787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986397, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.189@KRBTEST.COM for kudu/127.5.91.188@KRBTEST.COM
I20260620 20:13:17.613840 28026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4966ae8b2a9842a2a6b9d9be8b059df5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44805 } } peers { permanent_uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } } peers { permanent_uuid: "4966ae8b2a9842a2a6b9d9be8b059df5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } } }
I20260620 20:13:17.614005 28026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4966ae8b2a9842a2a6b9d9be8b059df5 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:13:17.615871 27868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997: Bootstrap starting.
I20260620 20:13:17.617484 27946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.608015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:39693 (local address 127.0.0.1:40912)
0620 20:13:17.608258 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:17.608272 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:13:17.608346 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:13:17.609034 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:13:17.609038 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:13:17.609207 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:13:17.609299 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:17.609304 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.610311 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.610315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:17.611120 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:17.611131 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.611843 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.611847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:17.611935 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.612390 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:13:17.612404 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:13:17.613968 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:13:17.616004 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:17.616008 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:17.616010 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:17.616249 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:17.616501 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:17.616504 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:17.616506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:17.616593 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:17.616966 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:13:17.616969 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:13:17.617082 (+ 113us) client_negotiation.cc:769] Sending connection context
0620 20:13:17.617189 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 20:13:17.617332 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170}
I20260620 20:13:17.617538 28025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.608466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40912 (local address 127.5.91.188:39693)
0620 20:13:17.608759 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.608767 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.608779 (+ 12us) server_negotiation.cc:407] Connection header received
0620 20:13:17.608809 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.608812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.608852 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.609012 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.609405 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.610203 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.611232 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.611733 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.611815 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.614230 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:17.614235 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:17.614238 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:17.614261 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:17.615892 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.616336 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.616339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.616341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.616420 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.616672 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.616674 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.616676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.616874 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:17.616961 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:17.617238 (+ 277us) server_negotiation.cc:299] Negotiation successful
0620 20:13:17.617353 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":117,"threads_started":1}
I20260620 20:13:17.618551 27868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997: Neither blocks nor log segments found. Creating new log.
I20260620 20:13:17.619234 27868 log.cc:824] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997: Log is configured to *not* fsync() on all Append() calls
I20260620 20:13:17.620018 27943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e7ff7a54c0f4a858ca615aa0a9ddab2: Bootstrap starting.
I20260620 20:13:17.620342 28019 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4966ae8b2a9842a2a6b9d9be8b059df5 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:13:17.621366 27868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997: No bootstrap required, opened a new log
I20260620 20:13:17.623184 27943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e7ff7a54c0f4a858ca615aa0a9ddab2: Neither blocks nor log segments found. Creating new log.
I20260620 20:13:17.624524 27868 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44805 } } peers { permanent_uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } } peers { permanent_uuid: "4966ae8b2a9842a2a6b9d9be8b059df5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } }
I20260620 20:13:17.624783 27868 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c4a1a0cb8f24ed089a457fc6ec1a997, State: Initialized, Role: FOLLOWER
I20260620 20:13:17.625100 27868 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44805 } } peers { permanent_uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } } peers { permanent_uuid: "4966ae8b2a9842a2a6b9d9be8b059df5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } }
W20260620 20:13:17.625499 28038 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4966ae8b2a9842a2a6b9d9be8b059df5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 20:13:17.625607 28038 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 20:13:17.626364 27868 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:13:17.626466 27842 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260620 20:13:17.626574 27842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:13:17.623976 27950 master_runner.cc:186] Error getting master registration for 127.5.91.190:44805: OK, instance_id { permanent_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" instance_seqno: 1781986397137092 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 20:13:17.627681 27950 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 20:13:17.627835 27943 log.cc:824] T 00000000000000000000000000000000 P 1e7ff7a54c0f4a858ca615aa0a9ddab2: Log is configured to *not* fsync() on all Append() calls
I20260620 20:13:17.627894 27842 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44805 } } peers { permanent_uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } } peers { permanent_uuid: "4966ae8b2a9842a2a6b9d9be8b059df5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } }
I20260620 20:13:17.628824 27842 leader_election.cc:288] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997 [CANDIDATE]: Term 1 election: Requested vote from peers 1e7ff7a54c0f4a858ca615aa0a9ddab2 (127.5.91.189:45661), 4966ae8b2a9842a2a6b9d9be8b059df5 (127.5.91.188:39693)
I20260620 20:13:17.630236 27943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e7ff7a54c0f4a858ca615aa0a9ddab2: No bootstrap required, opened a new log
W20260620 20:13:17.633679 27821 tablet.cc:2369] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:13:17.635141 27993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4966ae8b2a9842a2a6b9d9be8b059df5"
W20260620 20:13:17.635244 28052 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 20:13:17.635306 28052 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 20:13:17.635404 27993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4966ae8b2a9842a2a6b9d9be8b059df5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:13:17.635516 28040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44805 } } peers { permanent_uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } } peers { permanent_uuid: "4966ae8b2a9842a2a6b9d9be8b059df5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } } }
I20260620 20:13:17.635635 28040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:13:17.636581 27993 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4966ae8b2a9842a2a6b9d9be8b059df5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c4a1a0cb8f24ed089a457fc6ec1a997 in term 1.
I20260620 20:13:17.637120 27816 leader_election.cc:302] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4966ae8b2a9842a2a6b9d9be8b059df5, 5c4a1a0cb8f24ed089a457fc6ec1a997; no voters:
I20260620 20:13:17.637236 27943 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e7ff7a54c0f4a858ca615aa0a9ddab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44805 } } peers { permanent_uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } } peers { permanent_uuid: "4966ae8b2a9842a2a6b9d9be8b059df5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } }
I20260620 20:13:17.637512 27943 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e7ff7a54c0f4a858ca615aa0a9ddab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e7ff7a54c0f4a858ca615aa0a9ddab2, State: Initialized, Role: FOLLOWER
I20260620 20:13:17.637904 27943 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e7ff7a54c0f4a858ca615aa0a9ddab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44805 } } peers { permanent_uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } } peers { permanent_uuid: "4966ae8b2a9842a2a6b9d9be8b059df5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } }
I20260620 20:13:17.639437 27943 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e7ff7a54c0f4a858ca615aa0a9ddab2 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:13:17.639712 27917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2"
I20260620 20:13:17.639925 27917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e7ff7a54c0f4a858ca615aa0a9ddab2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:13:17.637398 27800 master_runner.cc:186] Error getting master registration for 127.5.91.189:45661: OK, instance_id { permanent_uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2" instance_seqno: 1781986397349959 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 20:13:17.640782 27800 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 20:13:17.640915 28040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:13:17.641108 28040 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997 [term 1 LEADER]: Becoming Leader. State: Replica: 5c4a1a0cb8f24ed089a457fc6ec1a997, State: Running, Role: LEADER
I20260620 20:13:17.641161 27917 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e7ff7a54c0f4a858ca615aa0a9ddab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c4a1a0cb8f24ed089a457fc6ec1a997 in term 1.
I20260620 20:13:17.641458 28040 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44805 } } peers { permanent_uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } } peers { permanent_uuid: "4966ae8b2a9842a2a6b9d9be8b059df5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } }
I20260620 20:13:17.643635 28040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c4a1a0cb8f24ed089a457fc6ec1a997. Latest consensus state: current_term: 1 leader_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44805 } } peers { permanent_uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } } peers { permanent_uuid: "4966ae8b2a9842a2a6b9d9be8b059df5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } } }
I20260620 20:13:17.643774 28040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997 [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:17.644157 28073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:13:17.645221 28073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260620 20:13:17.652060 28071 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1e7ff7a54c0f4a858ca615aa0a9ddab2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 20:13:17.652158 28071 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 20:13:17.652429 27993 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4966ae8b2a9842a2a6b9d9be8b059df5 [term 1 FOLLOWER]: Refusing update from remote peer 5c4a1a0cb8f24ed089a457fc6ec1a997: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:13:17.652410 28059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e7ff7a54c0f4a858ca615aa0a9ddab2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44805 } } peers { permanent_uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } } peers { permanent_uuid: "4966ae8b2a9842a2a6b9d9be8b059df5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } } }
I20260620 20:13:17.652524 28059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e7ff7a54c0f4a858ca615aa0a9ddab2 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:13:17.653129 28066 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4966ae8b2a9842a2a6b9d9be8b059df5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:13:17.653714 27917 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e7ff7a54c0f4a858ca615aa0a9ddab2 [term 1 FOLLOWER]: Refusing update from remote peer 5c4a1a0cb8f24ed089a457fc6ec1a997: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:13:17.657573 28026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4966ae8b2a9842a2a6b9d9be8b059df5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c4a1a0cb8f24ed089a457fc6ec1a997. Latest consensus state: current_term: 1 leader_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44805 } } peers { permanent_uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } } peers { permanent_uuid: "4966ae8b2a9842a2a6b9d9be8b059df5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } } }
I20260620 20:13:17.657722 28026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4966ae8b2a9842a2a6b9d9be8b059df5 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:13:17.657861 28066 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:13:17.658319 27872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.631339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39058 (local address 127.5.91.190:44805)
0620 20:13:17.631483 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.631488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.647699 (+ 16211us) server_negotiation.cc:407] Connection header received
0620 20:13:17.647880 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.647883 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.647932 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.647982 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.648967 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.649744 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.650961 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.651291 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.651360 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.654386 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:17.654390 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:17.654393 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:17.654420 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:17.656229 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.656745 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.656749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.656751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.656850 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.657202 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.657206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.657208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.657463 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:17.657582 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:17.658000 (+ 418us) server_negotiation.cc:299] Negotiation successful
0620 20:13:17.658123 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 20:13:17.661878 28059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e7ff7a54c0f4a858ca615aa0a9ddab2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c4a1a0cb8f24ed089a457fc6ec1a997. Latest consensus state: current_term: 1 leader_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44805 } } peers { permanent_uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } } peers { permanent_uuid: "4966ae8b2a9842a2a6b9d9be8b059df5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } } }
I20260620 20:13:17.662014 28059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e7ff7a54c0f4a858ca615aa0a9ddab2 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:13:17.668625 27949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.646508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46988 (local address 127.5.91.189:45661)
0620 20:13:17.646649 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.646654 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.653276 (+ 6622us) server_negotiation.cc:407] Connection header received
0620 20:13:17.653433 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.653436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.653481 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.653538 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.655915 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.656688 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.659697 (+ 3009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.660045 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.660114 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.664712 (+ 4598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:17.664716 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:17.664719 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:17.664747 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:17.666472 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.667013 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.667018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.667019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.667117 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.667446 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.667451 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.667453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.667698 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:17.667832 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:17.668327 (+ 495us) server_negotiation.cc:299] Negotiation successful
0620 20:13:17.668451 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 20:13:17.670003 28025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:17.647203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40924 (local address 127.5.91.188:39693)
0620 20:13:17.647322 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.647327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.654911 (+ 7584us) server_negotiation.cc:407] Connection header received
0620 20:13:17.655057 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.655060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.655108 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.655154 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.657754 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.658514 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.662693 (+ 4179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:17.663067 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:17.663139 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:17.666277 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:17.666281 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:17.666284 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:17.666312 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:17.667934 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.668383 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.668387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.668389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.668475 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:17.668765 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:17.668768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:17.668770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:17.668969 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:17.669093 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:17.669736 (+ 643us) server_negotiation.cc:299] Negotiation successful
0620 20:13:17.669855 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 20:13:17.671469 28066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44805 } } peers { permanent_uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } } peers { permanent_uuid: "4966ae8b2a9842a2a6b9d9be8b059df5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } } }
I20260620 20:13:17.671618 28066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997 [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:17.671732 28075 mvcc.cc:205] Tried to move back new op lower bound from 7299016284776620032 to 7299016284743225344. Current Snapshot: MvccSnapshot[applied={T|T < 7299016284776620032}]
I20260620 20:13:17.672835 28059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e7ff7a54c0f4a858ca615aa0a9ddab2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44805 } } peers { permanent_uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } } peers { permanent_uuid: "4966ae8b2a9842a2a6b9d9be8b059df5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } } }
I20260620 20:13:17.672948 28059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e7ff7a54c0f4a858ca615aa0a9ddab2 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:13:17.673286 28073 catalog_manager.cc:1347] Generated new cluster ID: 920d1f3240d74930862642aafdcbae17
I20260620 20:13:17.673345 28073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:13:17.673542 28080 mvcc.cc:205] Tried to move back new op lower bound from 7299016284776620032 to 7299016284743225344. Current Snapshot: MvccSnapshot[applied={T|T < 7299016284776620032}]
I20260620 20:13:17.685745 28040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44805 } } peers { permanent_uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } } peers { permanent_uuid: "4966ae8b2a9842a2a6b9d9be8b059df5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } } }
I20260620 20:13:17.685890 28040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997 [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:17.687006 28026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4966ae8b2a9842a2a6b9d9be8b059df5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4a1a0cb8f24ed089a457fc6ec1a997" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 44805 } } peers { permanent_uuid: "1e7ff7a54c0f4a858ca615aa0a9ddab2" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 45661 } } peers { permanent_uuid: "4966ae8b2a9842a2a6b9d9be8b059df5" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 39693 } } }
I20260620 20:13:17.687124 28026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4966ae8b2a9842a2a6b9d9be8b059df5 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:13:17.687778 28078 mvcc.cc:205] Tried to move back new op lower bound from 7299016284776620032 to 7299016284743225344. Current Snapshot: MvccSnapshot[applied={T|T < 7299016284776620032}]
I20260620 20:13:17.701390 28073 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:13:17.702713 28073 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:13:17.722493 28073 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c4a1a0cb8f24ed089a457fc6ec1a997: Generated new TSK 0
I20260620 20:13:17.723433 28073 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 20:13:17.761631 27872 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:17.758301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39060 (local address 127.5.91.190:44805)
0620 20:13:17.758452 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.758457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.760371 (+ 1914us) server_negotiation.cc:407] Connection header received
0620 20:13:17.760612 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.760616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.760703 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.760764 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.761472 (+ 708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39060: BlockingRecv error: recv got EOF from 127.0.0.1:39060 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260620 20:13:17.763198 27949 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:17.759158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47004 (local address 127.5.91.189:45661)
0620 20:13:17.759279 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.759284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.762234 (+ 2950us) server_negotiation.cc:407] Connection header received
0620 20:13:17.762438 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.762442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.762495 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.762550 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.763068 (+ 518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47004: BlockingRecv error: recv got EOF from 127.0.0.1:47004 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260620 20:13:17.765769 28025 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:17.759828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40930 (local address 127.5.91.188:39693)
0620 20:13:17.759975 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.759980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.763721 (+ 3741us) server_negotiation.cc:407] Connection header received
0620 20:13:17.763903 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.763906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.763960 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.764015 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.765646 (+ 1631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40930: BlockingRecv error: recv got EOF from 127.0.0.1:40930 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260620 20:13:17.769942 27949 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:17.768308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47006 (local address 127.5.91.189:45661)
0620 20:13:17.768434 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.768439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.768897 (+ 458us) server_negotiation.cc:407] Connection header received
0620 20:13:17.769120 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.769124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.769201 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.769259 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.769832 (+ 573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47006: BlockingRecv error: recv got EOF from 127.0.0.1:47006 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260620 20:13:17.787351 27949 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:17.774263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47008 (local address 127.5.91.189:45661)
0620 20:13:17.774397 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.774402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.774933 (+ 531us) server_negotiation.cc:407] Connection header received
0620 20:13:17.786485 (+ 11552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.786489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.786562 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.786615 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.787225 (+ 610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47008: BlockingRecv error: recv got EOF from 127.0.0.1:47008 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260620 20:13:17.803670 27949 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:17.791811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47018 (local address 127.5.91.189:45661)
0620 20:13:17.791927 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.791932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.798510 (+ 6578us) server_negotiation.cc:407] Connection header received
0620 20:13:17.798757 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.798762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.798835 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.798947 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.803538 (+ 4591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47018: BlockingRecv error: recv got EOF from 127.0.0.1:47018 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260620 20:13:17.811492 27949 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:17.809952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47030 (local address 127.5.91.189:45661)
0620 20:13:17.810107 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.810112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.810436 (+ 324us) server_negotiation.cc:407] Connection header received
0620 20:13:17.810656 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.810660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.810718 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.810778 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.811367 (+ 589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47030: BlockingRecv error: recv got EOF from 127.0.0.1:47030 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260620 20:13:17.824607 27949 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:17.822669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47040 (local address 127.5.91.189:45661)
0620 20:13:17.822805 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.822811 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.823451 (+ 640us) server_negotiation.cc:407] Connection header received
0620 20:13:17.823685 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.823689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.823748 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.823808 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.824472 (+ 664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47040: BlockingRecv error: recv got EOF from 127.0.0.1:47040 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260620 20:13:17.845739 27949 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:17.843804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47044 (local address 127.5.91.189:45661)
0620 20:13:17.843949 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.843955 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.844575 (+ 620us) server_negotiation.cc:407] Connection header received
0620 20:13:17.844824 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.844828 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.844893 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.844957 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.845603 (+ 646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47044: BlockingRecv error: recv got EOF from 127.0.0.1:47044 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260620 20:13:17.881381 27949 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:17.880295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47050 (local address 127.5.91.189:45661)
0620 20:13:17.880406 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.880411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.880584 (+ 173us) server_negotiation.cc:407] Connection header received
0620 20:13:17.880688 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.880690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.880740 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.880822 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.881263 (+ 441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47050: BlockingRecv error: recv got EOF from 127.0.0.1:47050 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260620 20:13:17.948714 27949 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:17.947500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47058 (local address 127.5.91.189:45661)
0620 20:13:17.947649 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:17.947653 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:17.947789 (+ 136us) server_negotiation.cc:407] Connection header received
0620 20:13:17.947912 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:17.947915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:17.947993 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:17.948065 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:17.948586 (+ 521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47058: BlockingRecv error: recv got EOF from 127.0.0.1:47058 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260620 20:13:18.080209 27949 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:18.078712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47064 (local address 127.5.91.189:45661)
0620 20:13:18.078858 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:18.078862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:18.078968 (+ 106us) server_negotiation.cc:407] Connection header received
0620 20:13:18.079262 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:18.079267 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:18.079353 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:18.079470 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:18.080035 (+ 565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47064: BlockingRecv error: recv got EOF from 127.0.0.1:47064 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260620 20:13:18.339668 27949 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:18.338231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47066 (local address 127.5.91.189:45661)
0620 20:13:18.338358 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:18.338361 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:18.338381 (+ 20us) server_negotiation.cc:407] Connection header received
0620 20:13:18.338603 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:18.338606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:18.338658 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:18.338751 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:18.339530 (+ 779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47066: BlockingRecv error: recv got EOF from 127.0.0.1:47066 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 20:13:18.627040 28038 catalog_manager.cc:1259] Loaded cluster ID: 920d1f3240d74930862642aafdcbae17
I20260620 20:13:18.627120 28038 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4966ae8b2a9842a2a6b9d9be8b059df5: loading cluster ID for follower catalog manager: success
I20260620 20:13:18.628718 28038 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4966ae8b2a9842a2a6b9d9be8b059df5: acquiring CA information for follower catalog manager: success
I20260620 20:13:18.629835 28038 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4966ae8b2a9842a2a6b9d9be8b059df5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 20:13:18.653434 28071 catalog_manager.cc:1259] Loaded cluster ID: 920d1f3240d74930862642aafdcbae17
I20260620 20:13:18.653532 28071 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1e7ff7a54c0f4a858ca615aa0a9ddab2: loading cluster ID for follower catalog manager: success
I20260620 20:13:18.655210 28071 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1e7ff7a54c0f4a858ca615aa0a9ddab2: acquiring CA information for follower catalog manager: success
I20260620 20:13:18.656327 28071 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1e7ff7a54c0f4a858ca615aa0a9ddab2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 20:13:18.857733 28149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:18.853698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47074 (local address 127.5.91.189:45661)
0620 20:13:18.853929 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:18.853932 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:18.854105 (+ 173us) server_negotiation.cc:407] Connection header received
0620 20:13:18.854235 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:18.854237 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:18.854364 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:18.854461 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:13:18.854753 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:18.855305 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:18.856225 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:18.856450 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:18.856529 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:18.856846 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:13:18.857288 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:13:18.857387 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:18.857477 (+ 90us) server_negotiation.cc:299] Negotiation successful
0620 20:13:18.857530 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":87,"threads_started":1}
I20260620 20:13:18.865093 28155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:18.860984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40932 (local address 127.5.91.188:39693)
0620 20:13:18.861337 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:18.861340 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:18.861353 (+ 13us) server_negotiation.cc:407] Connection header received
0620 20:13:18.861425 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:18.861427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:18.861537 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:18.861631 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:13:18.861967 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:18.862591 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:18.863546 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:18.863781 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:18.863860 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:18.864218 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:13:18.864657 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:13:18.864788 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:18.864888 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 20:13:18.864939 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":69,"threads_started":1}
I20260620 20:13:18.871903 28155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:18.868528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40946 (local address 127.5.91.188:39693)
0620 20:13:18.868625 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:18.868629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:18.868641 (+ 12us) server_negotiation.cc:407] Connection header received
0620 20:13:18.868730 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:18.868733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:18.868780 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:18.868836 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:13:18.869283 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:18.869806 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:18.870890 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:18.871155 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:18.871257 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:18.871487 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:13:18.871572 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:13:18.871633 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:18.871725 (+ 92us) server_negotiation.cc:299] Negotiation successful
0620 20:13:18.871774 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 20:13:18.873052 28162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:18.868244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39074 (local address 127.5.91.190:44805)
0620 20:13:18.868481 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:18.868485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:18.868500 (+ 15us) server_negotiation.cc:407] Connection header received
0620 20:13:18.868601 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:18.868604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:18.868781 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:18.868970 (+ 189us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:13:18.869333 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:18.870175 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:18.871155 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:18.871445 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:18.871540 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:18.872346 (+ 806us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:13:18.872689 (+ 343us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:13:18.872776 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:18.872851 (+ 75us) server_negotiation.cc:299] Negotiation successful
0620 20:13:18.872913 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":68,"threads_started":1}
I20260620 20:13:18.873143 28149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:18.868103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47086 (local address 127.5.91.189:45661)
0620 20:13:18.868243 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:18.868247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:18.868635 (+ 388us) server_negotiation.cc:407] Connection header received
0620 20:13:18.868799 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:18.868802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:18.868851 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:18.868915 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:13:18.869320 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:18.870163 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:18.872046 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:18.872342 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:18.872439 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:18.872679 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:13:18.872752 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:13:18.872810 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:18.872955 (+ 145us) server_negotiation.cc:299] Negotiation successful
0620 20:13:18.873014 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260620 20:13:18.880739 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 27800
I20260620 20:13:18.886061 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 27875
I20260620 20:13:18.890950 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 27950
2026-06-20T20:13:18Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3031 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 20:13:18 dist-test-slave-l8h4 krb5kdc[28166](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 20:13:18 dist-test-slave-l8h4 krb5kdc[28166](info): set up 2 sockets
Jun 20 20:13:18 dist-test-slave-l8h4 krb5kdc[28166](info): commencing operation
krb5kdc: starting...
W20260620 20:13:19.879179 5486 mini_kdc.cc:121] Time spent starting KDC: real 0.968s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/test-user.keytab.
Jun 20 20:13:19 dist-test-slave-l8h4 krb5kdc[28166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T20:13:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T20:13:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.91.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
Entry for principal kudu/127.5.91.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab.
I20260620 20:13:20.003005 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:46445
--webserver_interface=127.5.91.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:33265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:46445,127.5.91.189:33389,127.5.91.188:41955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 20:13:20.117761 28179 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 20:13:20.118208 28179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:13:20.118270 28179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:13:20.118333 28179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:13:20.122412 28179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:13:20.122467 28179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:13:20.122494 28179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:13:20.122516 28179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:13:20.122546 28179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:13:20.127826 28179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:33265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:46445,127.5.91.189:33389,127.5.91.188:41955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.190.keytab
--principal=kudu/127.5.91.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.190:46445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.91.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:13:20.128855 28179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:13:20.129828 28179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:13:20.137881 28184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:20.137887 28187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:20.137887 28185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:20.138269 28179 server_base.cc:1034] running on GCE node
I20260620 20:13:20.138816 28179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:13:20.139875 28179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:13:20.141036 28179 hybrid_clock.cc:648] HybridClock initialized: now 1781986400141022 us; error 26 us; skew 500 ppm
Jun 20 20:13:20 dist-test-slave-l8h4 krb5kdc[28166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:13:20.143787 28179 init.cc:377] Logged in from keytab as kudu/127.5.91.190@KRBTEST.COM (short username kudu)
I20260620 20:13:20.144675 28179 webserver.cc:460] Webserver started at http://127.5.91.190:42081/ using document root <none> and password file <none>
I20260620 20:13:20.145195 28179 fs_manager.cc:359] Metadata directory not provided
I20260620 20:13:20.145259 28179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:13:20.145439 28179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:13:20.147461 28179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-0/data/instance:
uuid: "e6cbc953e27142ddb10555952915e5f8"
format_stamp: "Formatted at 2026-06-20 20:13:20 on dist-test-slave-l8h4"
server_key: "f92868c7f54d55790af7fafc8466d50b"
server_key_iv: "0099854d389f4611f678325672338016"
server_key_version: "encryptionkey@0"
I20260620 20:13:20.148021 28179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-0/wal/instance:
uuid: "e6cbc953e27142ddb10555952915e5f8"
format_stamp: "Formatted at 2026-06-20 20:13:20 on dist-test-slave-l8h4"
server_key: "f92868c7f54d55790af7fafc8466d50b"
server_key_iv: "0099854d389f4611f678325672338016"
server_key_version: "encryptionkey@0"
I20260620 20:13:20.152312 28179 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 20:13:20.154894 28194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:13:20.156194 28179 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260620 20:13:20.156312 28179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-0/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-0/wal
uuid: "e6cbc953e27142ddb10555952915e5f8"
format_stamp: "Formatted at 2026-06-20 20:13:20 on dist-test-slave-l8h4"
server_key: "f92868c7f54d55790af7fafc8466d50b"
server_key_iv: "0099854d389f4611f678325672338016"
server_key_version: "encryptionkey@0"
I20260620 20:13:20.156412 28179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:13:20.189695 28179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:13:20.192898 28179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:13:20.193082 28179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:13:20.199719 28179 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.190:46445
I20260620 20:13:20.199967 28246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.190:46445 every 8 connection(s)
I20260620 20:13:20.200618 28179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-0/data/info.pb
I20260620 20:13:20.203213 28247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:13:20.206733 28247 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46445 } has no permanent_uuid. Determining permanent_uuid...
I20260620 20:13:20.210813 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 28179
I20260620 20:13:20.210934 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-0/wal/instance
I20260620 20:13:20.211212 5486 external_mini_cluster.cc:1428] Setting key d30242eddf677f5320ddd0d6ae4cff21
Jun 20 20:13:20 dist-test-slave-l8h4 krb5kdc[28166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.91.189@KRBTEST.COM; defaulting to no policy
I20260620 20:13:20.221390 28249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.208875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36280 (local address 127.5.91.190:46445)
0620 20:13:20.209336 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.209352 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.209528 (+ 176us) server_negotiation.cc:407] Connection header received
0620 20:13:20.210505 (+ 977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.210509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.210736 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.210929 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.211919 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.213041 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.214375 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.214727 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.214837 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.217258 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:20.217264 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:20.217273 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:20.217292 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:20.218736 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.219133 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.219139 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.219142 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.219236 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.219496 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.219499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.219501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.219900 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:20.220015 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:20.220346 (+ 331us) server_negotiation.cc:299] Negotiation successful
0620 20:13:20.220591 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":308,"thread_start_us":127,"threads_started":1}
I20260620 20:13:20.221401 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.208875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46445 (local address 127.0.0.1:36280)
0620 20:13:20.209383 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.209426 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 20:13:20.210235 (+ 809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:13:20.210982 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:13:20.210988 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:13:20.211424 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:13:20.211753 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:20.211769 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.213167 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.213170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:20.214265 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:20.214275 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.214876 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.214880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:20.215021 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.215571 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:13:20.215591 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:13:20.217134 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:13:20.218826 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:20.218834 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:20.218844 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:20.219042 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:20.219313 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:20.219317 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:20.219319 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:20.219416 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:20.219999 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:13:20.220005 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:13:20.220133 (+ 128us) client_negotiation.cc:769] Sending connection context
0620 20:13:20.220274 (+ 141us) client_negotiation.cc:240] Negotiation successful
0620 20:13:20.220721 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":116,"spinlock_wait_cycles":30976,"thread_start_us":166,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.189@KRBTEST.COM" created.
I20260620 20:13:20.223951 28247 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } has no permanent_uuid. Determining permanent_uuid...
I20260620 20:13:20.224670 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.224431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:33389 (local address 127.0.0.1:34744)
0620 20:13:20.224539 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.224613 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.189:33389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260620 20:13:20.224840 28196 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.91.189:33389: connect: Connection refused (error 111)
I20260620 20:13:20.225576 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.225396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:33389 (local address 127.0.0.1:34746)
0620 20:13:20.225490 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.225529 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.189:33389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260620 20:13:20.225697 28247 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.189:33389: Network error: Client connection negotiation failed: client connection to 127.5.91.189:33389: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.189.keytab.
Entry for principal kudu/127.5.91.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.189.keytab.
I20260620 20:13:20.238693 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.189:33389
--webserver_interface=127.5.91.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.189.keytab
--principal=kudu/127.5.91.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:33265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:46445,127.5.91.189:33389,127.5.91.188:41955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 20:13:20.274948 28247 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } attempt: 1
I20260620 20:13:20.275772 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.275470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:33389 (local address 127.0.0.1:34760)
0620 20:13:20.275640 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.275697 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.189:33389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260620 20:13:20.276453 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.276275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:33389 (local address 127.0.0.1:34768)
0620 20:13:20.276359 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.276400 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.189:33389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 20:13:20.276595 28247 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.189:33389: Network error: Client connection negotiation failed: client connection to 127.5.91.189:33389: connect: Connection refused (error 111)
I20260620 20:13:20.344807 28247 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } attempt: 2
I20260620 20:13:20.345705 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.345386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:33389 (local address 127.0.0.1:34770)
0620 20:13:20.345559 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.345625 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.189:33389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260620 20:13:20.346426 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.346226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:33389 (local address 127.0.0.1:34774)
0620 20:13:20.346322 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.346366 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.189:33389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 20:13:20.346623 28247 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.189:33389: Network error: Client connection negotiation failed: client connection to 127.5.91.189:33389: connect: Connection refused (error 111)
W20260620 20:13:20.351328 28254 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 20:13:20.351606 28254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:13:20.351650 28254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:13:20.351711 28254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:13:20.355772 28254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:13:20.355823 28254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:13:20.355849 28254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:13:20.355880 28254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:13:20.355911 28254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:13:20.361754 28254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:33265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:46445,127.5.91.189:33389,127.5.91.188:41955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.189.keytab
--principal=kudu/127.5.91.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.189:33389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.91.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:13:20.362847 28254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:13:20.363857 28254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:13:20.371942 28259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:20.372053 28254 server_base.cc:1034] running on GCE node
W20260620 20:13:20.371995 28260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:20.371995 28262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:20.372718 28254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:13:20.373723 28254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:13:20.374881 28254 hybrid_clock.cc:648] HybridClock initialized: now 1781986400374856 us; error 35 us; skew 500 ppm
Jun 20 20:13:20 dist-test-slave-l8h4 krb5kdc[28166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:13:20.377416 28254 init.cc:377] Logged in from keytab as kudu/127.5.91.189@KRBTEST.COM (short username kudu)
I20260620 20:13:20.378237 28254 webserver.cc:460] Webserver started at http://127.5.91.189:38023/ using document root <none> and password file <none>
I20260620 20:13:20.378726 28254 fs_manager.cc:359] Metadata directory not provided
I20260620 20:13:20.378779 28254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:13:20.378930 28254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:13:20.380836 28254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-1/data/instance:
uuid: "87e3effbf74c4af98f91a4caed3c5e02"
format_stamp: "Formatted at 2026-06-20 20:13:20 on dist-test-slave-l8h4"
server_key: "a14f3d458819b52d729ccbd88424263e"
server_key_iv: "50df128537326f4eabb950cdda68a1b2"
server_key_version: "encryptionkey@0"
I20260620 20:13:20.381311 28254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-1/wal/instance:
uuid: "87e3effbf74c4af98f91a4caed3c5e02"
format_stamp: "Formatted at 2026-06-20 20:13:20 on dist-test-slave-l8h4"
server_key: "a14f3d458819b52d729ccbd88424263e"
server_key_iv: "50df128537326f4eabb950cdda68a1b2"
server_key_version: "encryptionkey@0"
I20260620 20:13:20.384801 28254 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260620 20:13:20.387149 28269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:13:20.388716 28254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260620 20:13:20.388841 28254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-1/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-1/wal
uuid: "87e3effbf74c4af98f91a4caed3c5e02"
format_stamp: "Formatted at 2026-06-20 20:13:20 on dist-test-slave-l8h4"
server_key: "a14f3d458819b52d729ccbd88424263e"
server_key_iv: "50df128537326f4eabb950cdda68a1b2"
server_key_version: "encryptionkey@0"
I20260620 20:13:20.388945 28254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:13:20.404946 28254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:13:20.407990 28254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:13:20.408174 28254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:13:20.414722 28254 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.189:33389
I20260620 20:13:20.414969 28321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.189:33389 every 8 connection(s)
I20260620 20:13:20.415617 28254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-1/data/info.pb
I20260620 20:13:20.418290 28322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:13:20.422281 28322 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46445 } has no permanent_uuid. Determining permanent_uuid...
I20260620 20:13:20.425851 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 28254
I20260620 20:13:20.426021 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-1/wal/instance
I20260620 20:13:20.426375 5486 external_mini_cluster.cc:1428] Setting key 8b65176fa2339f0758b6e1f2ae0e0c14
Jun 20 20:13:20 dist-test-slave-l8h4 krb5kdc[28166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.189@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.91.188@KRBTEST.COM; defaulting to no policy
I20260620 20:13:20.436429 28249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.424099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36282 (local address 127.5.91.190:46445)
0620 20:13:20.424308 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.424311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.425211 (+ 900us) server_negotiation.cc:407] Connection header received
0620 20:13:20.426093 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.426096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.426188 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.426276 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.427410 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.428141 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.429413 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.429770 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.429887 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.432513 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:20.432517 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:20.432520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:20.432551 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:20.434206 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.434862 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.434866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.434868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.434950 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.435342 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.435345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.435346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.435542 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:20.435626 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:20.436197 (+ 571us) server_negotiation.cc:299] Negotiation successful
0620 20:13:20.436309 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.91.188@KRBTEST.COM" created.
I20260620 20:13:20.437469 28324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.424430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46445 (local address 127.0.0.1:36282)
0620 20:13:20.425036 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.425077 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 20:13:20.425887 (+ 810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:13:20.426428 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:13:20.426437 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:13:20.426804 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:13:20.427270 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:20.427282 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.428281 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.428285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:20.429275 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:20.429284 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.429944 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.429949 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:20.430123 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.430753 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:13:20.430768 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:13:20.432322 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:13:20.434326 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:20.434332 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:20.434344 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:20.434633 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:20.435103 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:20.435106 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:20.435107 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:20.435250 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:20.435641 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:13:20.435647 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:13:20.435890 (+ 243us) client_negotiation.cc:769] Sending connection context
0620 20:13:20.436136 (+ 246us) client_negotiation.cc:240] Negotiation successful
0620 20:13:20.436402 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":121,"threads_started":1}
I20260620 20:13:20.437783 28247 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } attempt: 3
I20260620 20:13:20.439167 28322 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 20:13:20 dist-test-slave-l8h4 krb5kdc[28166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for kudu/127.5.91.189@KRBTEST.COM
Jun 20 20:13:20 dist-test-slave-l8h4 krb5kdc[28166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.189@KRBTEST.COM for kudu/127.5.91.189@KRBTEST.COM
I20260620 20:13:20.448295 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.438199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:33389 (local address 127.0.0.1:34782)
0620 20:13:20.438361 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.438375 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:13:20.438466 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:13:20.438960 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:13:20.438963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:13:20.439132 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:13:20.439246 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:20.439254 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.440547 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.440550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:20.441788 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:20.441800 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.442476 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.442481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:20.442585 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.443225 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:13:20.443240 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:13:20.444508 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:13:20.446505 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:20.446508 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:20.446510 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:20.446726 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:20.447098 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:20.447100 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:20.447102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:20.447183 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:20.447733 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:13:20.447736 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:13:20.447842 (+ 106us) client_negotiation.cc:769] Sending connection context
0620 20:13:20.447976 (+ 134us) client_negotiation.cc:240] Negotiation successful
0620 20:13:20.448118 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260620 20:13:20.448438 28327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.438269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34782 (local address 127.5.91.189:33389)
0620 20:13:20.438524 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.438535 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.438554 (+ 19us) server_negotiation.cc:407] Connection header received
0620 20:13:20.438594 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.438611 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.438802 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.438932 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.439376 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.440443 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.441902 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.442368 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.442461 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.444669 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:20.444679 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:20.444694 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:20.444720 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:20.446405 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.446855 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.446861 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.446863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.446961 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.447276 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.447280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.447281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.447592 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:20.447741 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:20.448109 (+ 368us) server_negotiation.cc:299] Negotiation successful
0620 20:13:20.448262 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":164,"thread_start_us":85,"threads_started":1}
I20260620 20:13:20.449294 28324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.439649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:33389 (local address 127.0.0.1:34792)
0620 20:13:20.439792 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.439805 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:13:20.439869 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:13:20.440274 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:13:20.440276 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:13:20.440400 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:13:20.440592 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:20.440598 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.441348 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.441351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:20.441863 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:20.441872 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.442507 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.442510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:20.442853 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.443367 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:13:20.443380 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:13:20.445089 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:13:20.447684 (+ 2595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:20.447689 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:20.447691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:20.447922 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:20.448377 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:20.448381 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:20.448383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:20.448474 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:20.448820 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:13:20.448822 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:13:20.448899 (+ 77us) client_negotiation.cc:769] Sending connection context
0620 20:13:20.448983 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 20:13:20.449119 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":322}
I20260620 20:13:20.449419 28328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.439696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34792 (local address 127.5.91.189:33389)
0620 20:13:20.440053 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.440061 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.440073 (+ 12us) server_negotiation.cc:407] Connection header received
0620 20:13:20.440104 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.440107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.440143 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.440234 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.440709 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.441250 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.441958 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.442383 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.442820 (+ 437us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.445255 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:20.445259 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:20.445261 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:20.445288 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:20.447552 (+ 2264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.448027 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.448031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.448033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.448219 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.448564 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.448567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.448569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.448742 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:20.448829 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:20.449044 (+ 215us) server_negotiation.cc:299] Negotiation successful
0620 20:13:20.449205 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":152,"server-negotiator.queue_time_us":291,"thread_start_us":154,"threads_started":1}
I20260620 20:13:20.450577 28322 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } has no permanent_uuid. Determining permanent_uuid...
I20260620 20:13:20.450624 28247 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } has no permanent_uuid. Determining permanent_uuid...
I20260620 20:13:20.451262 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.451025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:41955 (local address 127.0.0.1:56682)
0620 20:13:20.451160 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.451205 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:41955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260620 20:13:20.451328 28324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.450995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:41955 (local address 127.0.0.1:56678)
0620 20:13:20.451155 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.451254 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:41955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260620 20:13:20.451495 28270 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.91.188:41955: connect: Connection refused (error 111)
I20260620 20:13:20.451977 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.451718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:41955 (local address 127.0.0.1:56686)
0620 20:13:20.451865 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.451915 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:41955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260620 20:13:20.452188 28247 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.188:41955: Network error: Client connection negotiation failed: client connection to 127.5.91.188:41955: connect: Connection refused (error 111)
I20260620 20:13:20.452288 28324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.452096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:41955 (local address 127.0.0.1:56692)
0620 20:13:20.452185 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.452233 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:41955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260620 20:13:20.452431 28322 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.188:41955: Network error: Client connection negotiation failed: client connection to 127.5.91.188:41955: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.91.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.188.keytab.
Entry for principal kudu/127.5.91.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.188.keytab.
I20260620 20:13:20.457657 5486 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
/tmp/dist-test-taskSTSRqO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.188:41955
--webserver_interface=127.5.91.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.188.keytab
--principal=kudu/127.5.91.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.91.148:33265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.91.190:46445,127.5.91.189:33389,127.5.91.188:41955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 20:13:20.483364 28247 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } attempt: 1
I20260620 20:13:20.484264 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.483919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:41955 (local address 127.0.0.1:56704)
0620 20:13:20.484092 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.484167 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:41955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260620 20:13:20.484956 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.484754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:41955 (local address 127.0.0.1:56716)
0620 20:13:20.484845 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.484894 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:41955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 20:13:20.485119 28247 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.188:41955: Network error: Client connection negotiation failed: client connection to 127.5.91.188:41955: connect: Connection refused (error 111)
I20260620 20:13:20.501706 28322 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } attempt: 1
I20260620 20:13:20.502677 28324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.502303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:41955 (local address 127.0.0.1:56726)
0620 20:13:20.502457 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.502546 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:41955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260620 20:13:20.503414 28324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.503203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:41955 (local address 127.0.0.1:56730)
0620 20:13:20.503304 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.503353 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:41955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260620 20:13:20.503580 28322 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.188:41955: Network error: Client connection negotiation failed: client connection to 127.5.91.188:41955: connect: Connection refused (error 111)
I20260620 20:13:20.560278 28247 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } attempt: 2
I20260620 20:13:20.561147 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.560810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:41955 (local address 127.0.0.1:56732)
0620 20:13:20.560940 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.561021 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:41955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260620 20:13:20.561839 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.561619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:41955 (local address 127.0.0.1:56744)
0620 20:13:20.561725 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.561776 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:41955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260620 20:13:20.561986 28247 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.188:41955: Network error: Client connection negotiation failed: client connection to 127.5.91.188:41955: connect: Connection refused (error 111)
W20260620 20:13:20.566447 28330 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 20:13:20.566709 28330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 20:13:20.566732 28330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 20:13:20.566768 28330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 20:13:20.570943 28330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 20:13:20.570995 28330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 20:13:20.571014 28330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 20:13:20.571030 28330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 20:13:20.571044 28330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 20:13:20.571748 28322 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } attempt: 2
I20260620 20:13:20.572466 28324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.572175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:41955 (local address 127.0.0.1:56748)
0620 20:13:20.572319 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.572381 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:41955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260620 20:13:20.573177 28324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.572967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:41955 (local address 127.0.0.1:56758)
0620 20:13:20.573067 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.573116 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.91.188:41955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 20:13:20.573302 28322 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.91.188:41955: Network error: Client connection negotiation failed: client connection to 127.5.91.188:41955: connect: Connection refused (error 111)
I20260620 20:13:20.576565 28330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.91.148:33265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.91.190:46445,127.5.91.189:33389,127.5.91.188:41955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/krb5kdc/kudu_127.5.91.188.keytab
--principal=kudu/127.5.91.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.91.188:41955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.91.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 19:21:02 UTC on ccycloud.quasar-kkgvxp.root.comops.site
build id 4457675
I20260620 20:13:20.577560 28330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 20:13:20.578572 28330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 20:13:20.586457 28335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 20:13:20.586431 28338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:20.586534 28330 server_base.cc:1034] running on GCE node
W20260620 20:13:20.586606 28336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 20:13:20.587210 28330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 20:13:20.588214 28330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 20:13:20.589371 28330 hybrid_clock.cc:648] HybridClock initialized: now 1781986400589356 us; error 26 us; skew 500 ppm
Jun 20 20:13:20 dist-test-slave-l8h4 krb5kdc[28166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 20:13:20.592029 28330 init.cc:377] Logged in from keytab as kudu/127.5.91.188@KRBTEST.COM (short username kudu)
I20260620 20:13:20.592857 28330 webserver.cc:460] Webserver started at http://127.5.91.188:44363/ using document root <none> and password file <none>
I20260620 20:13:20.593353 28330 fs_manager.cc:359] Metadata directory not provided
I20260620 20:13:20.593410 28330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 20:13:20.593595 28330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 20:13:20.595695 28330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-2/data/instance:
uuid: "6079bb11914247659fad00f0b13829b6"
format_stamp: "Formatted at 2026-06-20 20:13:20 on dist-test-slave-l8h4"
server_key: "3e2b6803ebfdf980e2b7cd09a5971656"
server_key_iv: "c4111ff309bc1d376b9009dbfdee34aa"
server_key_version: "encryptionkey@0"
I20260620 20:13:20.596259 28330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-2/wal/instance:
uuid: "6079bb11914247659fad00f0b13829b6"
format_stamp: "Formatted at 2026-06-20 20:13:20 on dist-test-slave-l8h4"
server_key: "3e2b6803ebfdf980e2b7cd09a5971656"
server_key_iv: "c4111ff309bc1d376b9009dbfdee34aa"
server_key_version: "encryptionkey@0"
I20260620 20:13:20.599776 28330 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 20:13:20.602195 28345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 20:13:20.603480 28330 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 20:13:20.603587 28330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-2/data,/tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-2/wal
uuid: "6079bb11914247659fad00f0b13829b6"
format_stamp: "Formatted at 2026-06-20 20:13:20 on dist-test-slave-l8h4"
server_key: "3e2b6803ebfdf980e2b7cd09a5971656"
server_key_iv: "c4111ff309bc1d376b9009dbfdee34aa"
server_key_version: "encryptionkey@0"
I20260620 20:13:20.603689 28330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 20:13:20.614357 28330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 20:13:20.617321 28330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 20:13:20.617483 28330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 20:13:20.623886 28330 rpc_server.cc:307] RPC server started. Bound to: 127.5.91.188:41955
I20260620 20:13:20.624189 28397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.91.188:41955 every 8 connection(s)
I20260620 20:13:20.624787 28330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-2/data/info.pb
I20260620 20:13:20.627455 28398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 20:13:20.631119 28398 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46445 } has no permanent_uuid. Determining permanent_uuid...
I20260620 20:13:20.634709 5486 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu as pid 28330
I20260620 20:13:20.634835 5486 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSTSRqO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781986111135528-5486-0/minicluster-data/master-2/wal/instance
I20260620 20:13:20.635274 5486 external_mini_cluster.cc:1428] Setting key 14014229c1d7d3aac89de7238fbd3c7c
Jun 20 20:13:20 dist-test-slave-l8h4 krb5kdc[28166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.188@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
Jun 20 20:13:20 dist-test-slave-l8h4 krb5kdc[28166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.188@KRBTEST.COM
I20260620 20:13:20.646351 28249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.633015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36286 (local address 127.5.91.190:46445)
0620 20:13:20.633163 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.633167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.634046 (+ 879us) server_negotiation.cc:407] Connection header received
0620 20:13:20.635167 (+ 1121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.635172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.635229 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.635324 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.636806 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.637562 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.639099 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.639422 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.639550 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.642332 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:20.642336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:20.642339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:20.642367 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:20.643775 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.644349 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.644352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.644354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.644428 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.644846 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.644850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.644851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.645097 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:20.645281 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:20.646063 (+ 782us) server_negotiation.cc:299] Negotiation successful
0620 20:13:20.646208 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260620 20:13:20.647411 28402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.635994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56772 (local address 127.5.91.188:41955)
0620 20:13:20.636278 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.636289 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.636337 (+ 48us) server_negotiation.cc:407] Connection header received
0620 20:13:20.636376 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.636380 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.636568 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.636727 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.637578 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.638924 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.639949 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.640343 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.640455 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.643094 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:20.643099 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:20.643109 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:20.643130 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:20.644734 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.645304 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.645309 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.645311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.645475 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.645933 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.645936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.645938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.646226 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:20.646418 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:20.646642 (+ 224us) server_negotiation.cc:299] Negotiation successful
0620 20:13:20.646780 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":197,"thread_start_us":103,"threads_started":1}
I20260620 20:13:20.647411 28400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.633304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.190:46445 (local address 127.0.0.1:36286)
0620 20:13:20.633848 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.633893 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 20:13:20.634805 (+ 912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:13:20.635542 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:13:20.635549 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:13:20.636022 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:13:20.636648 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:20.636664 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.637673 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.637676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:20.638935 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:20.638941 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.639666 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.639672 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:20.639824 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.640554 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:13:20.640573 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:13:20.642187 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:13:20.643883 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:20.643890 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:20.643899 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:20.644235 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:20.644510 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:20.644514 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:20.644516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:20.644691 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:20.645289 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:13:20.645296 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:13:20.645803 (+ 507us) client_negotiation.cc:769] Sending connection context
0620 20:13:20.645976 (+ 173us) client_negotiation.cc:240] Negotiation successful
0620 20:13:20.646317 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":37,"thread_start_us":132,"threads_started":1}
I20260620 20:13:20.649331 28398 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 20:13:20 dist-test-slave-l8h4 krb5kdc[28166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.188@KRBTEST.COM for kudu/127.5.91.189@KRBTEST.COM
Jun 20 20:13:20 dist-test-slave-l8h4 krb5kdc[28166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.190@KRBTEST.COM
I20260620 20:13:20.658043 28400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.649829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.189:33389 (local address 127.0.0.1:34806)
0620 20:13:20.650020 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.650047 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0620 20:13:20.650184 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:13:20.650403 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:13:20.650406 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:13:20.650555 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:13:20.650646 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:20.650651 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.651500 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.651504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:20.652035 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:20.652044 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.652552 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.652555 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:20.652682 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.653074 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:13:20.653086 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:13:20.654496 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:13:20.656707 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:20.656710 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:20.656712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:20.656906 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:20.657154 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:20.657157 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:20.657159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:20.657236 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:20.657576 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:13:20.657579 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:13:20.657666 (+ 87us) client_negotiation.cc:769] Sending connection context
0620 20:13:20.657748 (+ 82us) client_negotiation.cc:240] Negotiation successful
0620 20:13:20.657878 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260620 20:13:20.658053 28328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.649961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34806 (local address 127.5.91.189:33389)
0620 20:13:20.650131 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.650157 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.650172 (+ 15us) server_negotiation.cc:407] Connection header received
0620 20:13:20.650272 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.650275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.650322 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.650381 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.650751 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.651383 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.652180 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.652436 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.652539 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.654667 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:20.654670 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:20.654672 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:20.654694 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:20.656580 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.656994 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.656998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.657000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.657074 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.657313 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.657316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.657318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.657493 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:20.657574 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:20.657809 (+ 235us) server_negotiation.cc:299] Negotiation successful
0620 20:13:20.657922 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260620 20:13:20.658381 28249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.649960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36290 (local address 127.5.91.190:46445)
0620 20:13:20.650267 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.650271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.650282 (+ 11us) server_negotiation.cc:407] Connection header received
0620 20:13:20.650311 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.650313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.650349 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.650409 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.650977 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.651459 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.652153 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.652353 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.652433 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.655146 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:20.655150 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:20.655152 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:20.655176 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:20.656945 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.657307 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.657311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.657312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.657385 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.657622 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.657625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.657627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.657792 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:20.657982 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:20.658112 (+ 130us) server_negotiation.cc:299] Negotiation successful
0620 20:13:20.658241 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228}
I20260620 20:13:20.659107 28398 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } has no permanent_uuid. Determining permanent_uuid...
I20260620 20:13:20.661135 28247 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } attempt: 3
Jun 20 20:13:20 dist-test-slave-l8h4 krb5kdc[28166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.188@KRBTEST.COM for kudu/127.5.91.188@KRBTEST.COM
I20260620 20:13:20.664454 28322 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } attempt: 3
Jun 20 20:13:20 dist-test-slave-l8h4 krb5kdc[28166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.91.189@KRBTEST.COM
Jun 20 20:13:20 dist-test-slave-l8h4 krb5kdc[28166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.190@KRBTEST.COM for kudu/127.5.91.188@KRBTEST.COM
I20260620 20:13:20.668012 28400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.659638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:41955 (local address 127.0.0.1:56784)
0620 20:13:20.659732 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.659745 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 20:13:20.659813 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:13:20.660054 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:13:20.660058 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:13:20.660215 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:13:20.660289 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:20.660295 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.661243 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.661246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:20.661835 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:20.661843 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.662426 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.662429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:20.662496 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.662881 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:13:20.662894 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:13:20.664183 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:13:20.666574 (+ 2391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:20.666577 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:20.666578 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:20.666791 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:20.667057 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:20.667059 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:20.667061 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:20.667135 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:20.667564 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:13:20.667566 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:13:20.667638 (+ 72us) client_negotiation.cc:769] Sending connection context
0620 20:13:20.667736 (+ 98us) client_negotiation.cc:240] Negotiation successful
0620 20:13:20.667850 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"spinlock_wait_cycles":3968}
I20260620 20:13:20.668654 28402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.659699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56784 (local address 127.5.91.188:41955)
0620 20:13:20.659836 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.659841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.659858 (+ 17us) server_negotiation.cc:407] Connection header received
0620 20:13:20.659895 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.659899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.659944 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.660024 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.660392 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.661147 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.661936 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.662241 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.662332 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.664323 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:20.664325 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:20.664327 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:20.664346 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:20.666483 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.666886 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.666889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.666890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.666964 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.667272 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.667275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.667276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.667457 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:20.667536 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:20.668401 (+ 865us) server_negotiation.cc:299] Negotiation successful
0620 20:13:20.668523 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 20:13:20.670454 28328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.660080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34814 (local address 127.5.91.189:33389)
0620 20:13:20.660175 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.660180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.660192 (+ 12us) server_negotiation.cc:407] Connection header received
0620 20:13:20.660228 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.660231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.660267 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.660334 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.661061 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.662232 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.663228 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.663514 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.663611 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.666554 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:20.666559 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:20.666562 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:20.666589 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:20.668240 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.668741 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.668745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.668746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.668841 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.669360 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.669364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.669366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.669590 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:20.669756 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:20.670163 (+ 407us) server_negotiation.cc:299] Negotiation successful
0620 20:13:20.670295 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
Jun 20 20:13:20 dist-test-slave-l8h4 krb5kdc[28166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781986400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.91.189@KRBTEST.COM for kudu/127.5.91.188@KRBTEST.COM
I20260620 20:13:20.671633 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.661463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:41955 (local address 127.0.0.1:56786)
0620 20:13:20.661774 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.661789 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 20:13:20.661871 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:13:20.662118 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:13:20.662121 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:13:20.662329 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:13:20.662422 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:20.662428 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.663469 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.663472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:20.664153 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:20.664164 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.665578 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.665583 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:20.665679 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.666050 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:13:20.666066 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:13:20.667461 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:13:20.669709 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:20.669712 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:20.669714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:20.669958 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:20.670331 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:20.670335 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:20.670337 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:20.670438 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:20.671128 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:13:20.671132 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:13:20.671235 (+ 103us) client_negotiation.cc:769] Sending connection context
0620 20:13:20.671324 (+ 89us) client_negotiation.cc:240] Negotiation successful
0620 20:13:20.671469 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238}
I20260620 20:13:20.672246 28398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6079bb11914247659fad00f0b13829b6: Bootstrap starting.
I20260620 20:13:20.672259 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.661527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56786 (local address 127.5.91.188:41955)
0620 20:13:20.661830 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.661840 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.661854 (+ 14us) server_negotiation.cc:407] Connection header received
0620 20:13:20.661965 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.661968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.662007 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.662088 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.662535 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.663363 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.664264 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.665434 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.665887 (+ 453us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.667622 (+ 1735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:20.667626 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:20.667629 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:20.667656 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:20.669585 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.670087 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.670090 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.670092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.670214 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.670624 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.670627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.670629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.671012 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:20.671905 (+ 893us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:20.671980 (+ 75us) server_negotiation.cc:299] Negotiation successful
0620 20:13:20.672089 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":105,"threads_started":1}
I20260620 20:13:20.673992 5486 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 20:13:20.674321 28324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.664831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.91.188:41955 (local address 127.0.0.1:56790)
0620 20:13:20.664967 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 20:13:20.664981 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 20:13:20.665056 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 20:13:20.665567 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 20:13:20.665571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 20:13:20.666165 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0620 20:13:20.666262 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:20.666269 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.667785 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.667788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:20.668524 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 20:13:20.668533 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.669278 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.669281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 20:13:20.669356 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.669731 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 20:13:20.669744 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 20:13:20.670992 (+ 1248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 20:13:20.672867 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:20.672870 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:20.672872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:20.673084 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:20.673389 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 20:13:20.673393 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 20:13:20.673395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 20:13:20.673482 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 20:13:20.673892 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 20:13:20.673895 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 20:13:20.673980 (+ 85us) client_negotiation.cc:769] Sending connection context
0620 20:13:20.674051 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 20:13:20.674187 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260620 20:13:20.674865 28404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.665058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56790 (local address 127.5.91.188:41955)
0620 20:13:20.665321 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.665329 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.665340 (+ 11us) server_negotiation.cc:407] Connection header received
0620 20:13:20.665368 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.665371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.665408 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.666368 (+ 960us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.666481 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.667680 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.668662 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.669162 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.671112 (+ 1950us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.671205 (+ 93us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:20.671208 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:20.671211 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:20.671226 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:20.672752 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.673195 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.673198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.673199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.673272 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.673601 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.673604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.673605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.673788 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:20.674557 (+ 769us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:20.674625 (+ 68us) server_negotiation.cc:299] Negotiation successful
0620 20:13:20.674723 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":102,"threads_started":1}
I20260620 20:13:20.675139 28398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6079bb11914247659fad00f0b13829b6: Neither blocks nor log segments found. Creating new log.
I20260620 20:13:20.675199 28247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8: Bootstrap starting.
I20260620 20:13:20.675985 28398 log.cc:824] T 00000000000000000000000000000000 P 6079bb11914247659fad00f0b13829b6: Log is configured to *not* fsync() on all Append() calls
I20260620 20:13:20.675199 28221 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e6cbc953e27142ddb10555952915e5f8"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36290
I20260620 20:13:20.677413 28322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87e3effbf74c4af98f91a4caed3c5e02: Bootstrap starting.
I20260620 20:13:20.677680 28247 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8: Neither blocks nor log segments found. Creating new log.
I20260620 20:13:20.678503 28247 log.cc:824] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8: Log is configured to *not* fsync() on all Append() calls
I20260620 20:13:20.679304 28322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87e3effbf74c4af98f91a4caed3c5e02: Neither blocks nor log segments found. Creating new log.
I20260620 20:13:20.680088 28322 log.cc:824] T 00000000000000000000000000000000 P 87e3effbf74c4af98f91a4caed3c5e02: Log is configured to *not* fsync() on all Append() calls
I20260620 20:13:20.680647 28398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6079bb11914247659fad00f0b13829b6: No bootstrap required, opened a new log
I20260620 20:13:20.680660 28247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8: No bootstrap required, opened a new log
I20260620 20:13:20.681945 28322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87e3effbf74c4af98f91a4caed3c5e02: No bootstrap required, opened a new log
I20260620 20:13:20.683636 28247 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbc953e27142ddb10555952915e5f8" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46445 } } peers { permanent_uuid: "87e3effbf74c4af98f91a4caed3c5e02" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } } peers { permanent_uuid: "6079bb11914247659fad00f0b13829b6" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } }
I20260620 20:13:20.683858 28247 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6cbc953e27142ddb10555952915e5f8, State: Initialized, Role: FOLLOWER
I20260620 20:13:20.684171 28247 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbc953e27142ddb10555952915e5f8" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46445 } } peers { permanent_uuid: "87e3effbf74c4af98f91a4caed3c5e02" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } } peers { permanent_uuid: "6079bb11914247659fad00f0b13829b6" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } }
I20260620 20:13:20.684702 28408 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbc953e27142ddb10555952915e5f8" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46445 } } peers { permanent_uuid: "87e3effbf74c4af98f91a4caed3c5e02" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } } peers { permanent_uuid: "6079bb11914247659fad00f0b13829b6" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } } }
I20260620 20:13:20.684937 28408 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:13:20.684962 28398 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6079bb11914247659fad00f0b13829b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbc953e27142ddb10555952915e5f8" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46445 } } peers { permanent_uuid: "87e3effbf74c4af98f91a4caed3c5e02" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } } peers { permanent_uuid: "6079bb11914247659fad00f0b13829b6" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } }
I20260620 20:13:20.685189 28398 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6079bb11914247659fad00f0b13829b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6079bb11914247659fad00f0b13829b6, State: Initialized, Role: FOLLOWER
I20260620 20:13:20.685495 28247 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:13:20.685467 28398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6079bb11914247659fad00f0b13829b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbc953e27142ddb10555952915e5f8" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46445 } } peers { permanent_uuid: "87e3effbf74c4af98f91a4caed3c5e02" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } } peers { permanent_uuid: "6079bb11914247659fad00f0b13829b6" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } }
I20260620 20:13:20.685545 28221 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260620 20:13:20.685642 28221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:13:20.686282 28322 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87e3effbf74c4af98f91a4caed3c5e02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbc953e27142ddb10555952915e5f8" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46445 } } peers { permanent_uuid: "87e3effbf74c4af98f91a4caed3c5e02" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } } peers { permanent_uuid: "6079bb11914247659fad00f0b13829b6" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } }
I20260620 20:13:20.686540 28322 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87e3effbf74c4af98f91a4caed3c5e02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87e3effbf74c4af98f91a4caed3c5e02, State: Initialized, Role: FOLLOWER
I20260620 20:13:20.686738 28398 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6079bb11914247659fad00f0b13829b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:13:20.686924 28221 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbc953e27142ddb10555952915e5f8" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46445 } } peers { permanent_uuid: "87e3effbf74c4af98f91a4caed3c5e02" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } } peers { permanent_uuid: "6079bb11914247659fad00f0b13829b6" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } }
I20260620 20:13:20.686911 28322 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87e3effbf74c4af98f91a4caed3c5e02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbc953e27142ddb10555952915e5f8" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46445 } } peers { permanent_uuid: "87e3effbf74c4af98f91a4caed3c5e02" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } } peers { permanent_uuid: "6079bb11914247659fad00f0b13829b6" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } }
I20260620 20:13:20.687911 28221 leader_election.cc:288] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8 [CANDIDATE]: Term 1 election: Requested vote from peers 87e3effbf74c4af98f91a4caed3c5e02 (127.5.91.189:33389), 6079bb11914247659fad00f0b13829b6 (127.5.91.188:41955)
I20260620 20:13:20.688633 28322 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87e3effbf74c4af98f91a4caed3c5e02 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 20:13:20.689401 28295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e6cbc953e27142ddb10555952915e5f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "87e3effbf74c4af98f91a4caed3c5e02"
I20260620 20:13:20.689622 28295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87e3effbf74c4af98f91a4caed3c5e02 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:13:20.690980 28295 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 87e3effbf74c4af98f91a4caed3c5e02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6cbc953e27142ddb10555952915e5f8 in term 1.
W20260620 20:13:20.691368 28424 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 20:13:20.691461 28424 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 20:13:20.691459 28372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e6cbc953e27142ddb10555952915e5f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6079bb11914247659fad00f0b13829b6"
I20260620 20:13:20.691710 28372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6079bb11914247659fad00f0b13829b6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 20:13:20.691818 28418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e3effbf74c4af98f91a4caed3c5e02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbc953e27142ddb10555952915e5f8" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46445 } } peers { permanent_uuid: "87e3effbf74c4af98f91a4caed3c5e02" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } } peers { permanent_uuid: "6079bb11914247659fad00f0b13829b6" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } } }
I20260620 20:13:20.691957 28418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e3effbf74c4af98f91a4caed3c5e02 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:13:20.693377 28372 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6079bb11914247659fad00f0b13829b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6cbc953e27142ddb10555952915e5f8 in term 1.
I20260620 20:13:20.693856 28195 leader_election.cc:302] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6079bb11914247659fad00f0b13829b6, e6cbc953e27142ddb10555952915e5f8; no voters:
I20260620 20:13:20.694234 28408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 20:13:20.694494 28408 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8 [term 1 LEADER]: Becoming Leader. State: Replica: e6cbc953e27142ddb10555952915e5f8, State: Running, Role: LEADER
W20260620 20:13:20.694617 28200 tablet.cc:2369] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 20:13:20.694912 28408 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbc953e27142ddb10555952915e5f8" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46445 } } peers { permanent_uuid: "87e3effbf74c4af98f91a4caed3c5e02" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } } peers { permanent_uuid: "6079bb11914247659fad00f0b13829b6" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } }
I20260620 20:13:20.697844 28408 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6cbc953e27142ddb10555952915e5f8. Latest consensus state: current_term: 1 leader_uuid: "e6cbc953e27142ddb10555952915e5f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbc953e27142ddb10555952915e5f8" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46445 } } peers { permanent_uuid: "87e3effbf74c4af98f91a4caed3c5e02" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } } peers { permanent_uuid: "6079bb11914247659fad00f0b13829b6" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } } }
W20260620 20:13:20.697945 28432 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6079bb11914247659fad00f0b13829b6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 20:13:20.698025 28432 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 20:13:20.698004 28408 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8 [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:20.698508 28451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 20:13:20.696147 28330 master_runner.cc:186] Error getting master registration for 127.5.91.189:33389: OK, instance_id { permanent_uuid: "87e3effbf74c4af98f91a4caed3c5e02" instance_seqno: 1781986400412929 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 20:13:20.699218 28330 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 20:13:20.699589 28451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 20:13:20.700827 28409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6079bb11914247659fad00f0b13829b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbc953e27142ddb10555952915e5f8" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46445 } } peers { permanent_uuid: "87e3effbf74c4af98f91a4caed3c5e02" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } } peers { permanent_uuid: "6079bb11914247659fad00f0b13829b6" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } } }
I20260620 20:13:20.700943 28409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6079bb11914247659fad00f0b13829b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:13:20.700083 28179 master_runner.cc:186] Error getting master registration for 127.5.91.189:33389: OK, instance_id { permanent_uuid: "87e3effbf74c4af98f91a4caed3c5e02" instance_seqno: 1781986400412929 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 20:13:20.703912 28179 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 20:13:20.711135 28372 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6079bb11914247659fad00f0b13829b6 [term 1 FOLLOWER]: Refusing update from remote peer e6cbc953e27142ddb10555952915e5f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:13:20.711788 28408 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6079bb11914247659fad00f0b13829b6" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:13:20.715468 28409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6079bb11914247659fad00f0b13829b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6cbc953e27142ddb10555952915e5f8. Latest consensus state: current_term: 1 leader_uuid: "e6cbc953e27142ddb10555952915e5f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbc953e27142ddb10555952915e5f8" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46445 } } peers { permanent_uuid: "87e3effbf74c4af98f91a4caed3c5e02" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } } peers { permanent_uuid: "6079bb11914247659fad00f0b13829b6" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } } }
I20260620 20:13:20.715615 28409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6079bb11914247659fad00f0b13829b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:13:20.716516 28295 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 87e3effbf74c4af98f91a4caed3c5e02 [term 1 FOLLOWER]: Refusing update from remote peer e6cbc953e27142ddb10555952915e5f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 20:13:20.717053 28408 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87e3effbf74c4af98f91a4caed3c5e02" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 20:13:20.720804 28418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e3effbf74c4af98f91a4caed3c5e02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6cbc953e27142ddb10555952915e5f8. Latest consensus state: current_term: 1 leader_uuid: "e6cbc953e27142ddb10555952915e5f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbc953e27142ddb10555952915e5f8" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46445 } } peers { permanent_uuid: "87e3effbf74c4af98f91a4caed3c5e02" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } } peers { permanent_uuid: "6079bb11914247659fad00f0b13829b6" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } } }
I20260620 20:13:20.720923 28418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e3effbf74c4af98f91a4caed3c5e02 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 20:13:20.723830 28455 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 87e3effbf74c4af98f91a4caed3c5e02: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260620 20:13:20.725346 28409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6079bb11914247659fad00f0b13829b6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e6cbc953e27142ddb10555952915e5f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbc953e27142ddb10555952915e5f8" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46445 } } peers { permanent_uuid: "87e3effbf74c4af98f91a4caed3c5e02" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } } peers { permanent_uuid: "6079bb11914247659fad00f0b13829b6" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } } }
I20260620 20:13:20.725463 28409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6079bb11914247659fad00f0b13829b6 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 20:13:20.725561 28455 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 20:13:20.725827 28457 mvcc.cc:205] Tried to move back new op lower bound from 7299016297299546112 to 7299016297250213888. Current Snapshot: MvccSnapshot[applied={T|T < 7299016297299546112}]
I20260620 20:13:20.725502 28446 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e6cbc953e27142ddb10555952915e5f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbc953e27142ddb10555952915e5f8" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46445 } } peers { permanent_uuid: "87e3effbf74c4af98f91a4caed3c5e02" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } } peers { permanent_uuid: "6079bb11914247659fad00f0b13829b6" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } } }
I20260620 20:13:20.727034 28446 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8 [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:20.727525 28249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.698887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36306 (local address 127.5.91.190:46445)
0620 20:13:20.698985 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.698989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.699933 (+ 944us) server_negotiation.cc:407] Connection header received
0620 20:13:20.708228 (+ 8295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.708232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.708281 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.708329 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.712531 (+ 4202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.713272 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.715586 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.715921 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.715992 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.719472 (+ 3480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:20.719476 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:20.719479 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:20.719506 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:20.721137 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.721634 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.721638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.721640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.721723 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.724138 (+ 2415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.724142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.724144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.724366 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:20.724474 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:20.727241 (+ 2767us) server_negotiation.cc:299] Negotiation successful
0620 20:13:20.727365 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 20:13:20.729761 28408 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e6cbc953e27142ddb10555952915e5f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbc953e27142ddb10555952915e5f8" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46445 } } peers { permanent_uuid: "87e3effbf74c4af98f91a4caed3c5e02" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } } peers { permanent_uuid: "6079bb11914247659fad00f0b13829b6" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } } }
I20260620 20:13:20.729877 28408 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8 [sys.catalog]: This master's current role is: LEADER
I20260620 20:13:20.731081 28418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e3effbf74c4af98f91a4caed3c5e02 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e6cbc953e27142ddb10555952915e5f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbc953e27142ddb10555952915e5f8" member_type: VOTER last_known_addr { host: "127.5.91.190" port: 46445 } } peers { permanent_uuid: "87e3effbf74c4af98f91a4caed3c5e02" member_type: VOTER last_known_addr { host: "127.5.91.189" port: 33389 } } peers { permanent_uuid: "6079bb11914247659fad00f0b13829b6" member_type: VOTER last_known_addr { host: "127.5.91.188" port: 41955 } } }
I20260620 20:13:20.731196 28418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e3effbf74c4af98f91a4caed3c5e02 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 20:13:20.731652 28459 mvcc.cc:205] Tried to move back new op lower bound from 7299016297299546112 to 7299016297250213888. Current Snapshot: MvccSnapshot[applied={T|T < 7299016297299546112}]
I20260620 20:13:20.732653 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.698478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56806 (local address 127.5.91.188:41955)
0620 20:13:20.698590 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.698594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.710367 (+ 11773us) server_negotiation.cc:407] Connection header received
0620 20:13:20.710466 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.710469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.710516 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.710582 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.722906 (+ 12324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.723702 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.724774 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.725094 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.725171 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.726725 (+ 1554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:20.726729 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:20.726731 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:20.726766 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:20.728334 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.729391 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.729394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.729396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.729494 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.731119 (+ 1625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.731122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.731124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.731355 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:20.732036 (+ 681us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:20.732366 (+ 330us) server_negotiation.cc:299] Negotiation successful
0620 20:13:20.732489 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 20:13:20.733582 28328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:20.698544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34828 (local address 127.5.91.189:33389)
0620 20:13:20.698659 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.698664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.699742 (+ 1078us) server_negotiation.cc:407] Connection header received
0620 20:13:20.709322 (+ 9580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.709325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.709374 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.709421 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.710582 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.711357 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.714417 (+ 3060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:20.714767 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:20.714837 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:20.722044 (+ 7207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 20:13:20.722048 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 20:13:20.722051 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 20:13:20.722079 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 20:13:20.723653 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.727781 (+ 4128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.727786 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.727787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.727882 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 20:13:20.732633 (+ 4751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 20:13:20.732637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 20:13:20.732639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 20:13:20.732873 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 20:13:20.732985 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:20.733327 (+ 342us) server_negotiation.cc:299] Negotiation successful
0620 20:13:20.733450 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 20:13:20.734400 28451 catalog_manager.cc:1347] Generated new cluster ID: 8c0237d7b2f84a36ae7eb3d16a07fc2f
I20260620 20:13:20.734467 28451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 20:13:20.770431 28451 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 20:13:20.772281 28451 catalog_manager.cc:1504] Loading token signing keys...
I20260620 20:13:20.807147 28451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6cbc953e27142ddb10555952915e5f8: Generated new TSK 0
I20260620 20:13:20.810595 28451 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 20:13:20.854722 28249 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:20.852187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36310 (local address 127.5.91.190:46445)
0620 20:13:20.852325 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.852330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.853620 (+ 1290us) server_negotiation.cc:407] Connection header received
0620 20:13:20.853816 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.853820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.853868 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.853917 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.854585 (+ 668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36310: BlockingRecv error: recv got EOF from 127.0.0.1:36310 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260620 20:13:20.856112 28328 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:20.852988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34836 (local address 127.5.91.189:33389)
0620 20:13:20.853105 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.853109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.855221 (+ 2112us) server_negotiation.cc:407] Connection header received
0620 20:13:20.855399 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.855402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.855454 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.855503 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.855987 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34836: BlockingRecv error: recv got EOF from 127.0.0.1:34836 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260620 20:13:20.858008 28403 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:20.856759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56820 (local address 127.5.91.188:41955)
0620 20:13:20.856865 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.856870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.857079 (+ 209us) server_negotiation.cc:407] Connection header received
0620 20:13:20.857195 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.857198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.857248 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.857316 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.857885 (+ 569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56820: BlockingRecv error: recv got EOF from 127.0.0.1:56820 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260620 20:13:20.863183 28328 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:20.861490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34852 (local address 127.5.91.189:33389)
0620 20:13:20.861612 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.861617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.862160 (+ 543us) server_negotiation.cc:407] Connection header received
0620 20:13:20.862398 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.862401 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.862482 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.862534 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.863072 (+ 538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34852: BlockingRecv error: recv got EOF from 127.0.0.1:34852 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260620 20:13:20.872252 28328 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:20.870495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34864 (local address 127.5.91.189:33389)
0620 20:13:20.870636 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.870641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.871278 (+ 637us) server_negotiation.cc:407] Connection header received
0620 20:13:20.871475 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.871478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.871544 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.871593 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.872133 (+ 540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34864: BlockingRecv error: recv got EOF from 127.0.0.1:34864 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260620 20:13:20.878320 28328 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:20.876654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34876 (local address 127.5.91.189:33389)
0620 20:13:20.876768 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.876772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.877328 (+ 556us) server_negotiation.cc:407] Connection header received
0620 20:13:20.877539 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.877543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.877599 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.877653 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.878213 (+ 560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34876: BlockingRecv error: recv got EOF from 127.0.0.1:34876 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260620 20:13:20.886356 28328 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:20.884657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34888 (local address 127.5.91.189:33389)
0620 20:13:20.884796 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.884801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.885356 (+ 555us) server_negotiation.cc:407] Connection header received
0620 20:13:20.885569 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.885573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.885629 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.885683 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.886247 (+ 564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34888: BlockingRecv error: recv got EOF from 127.0.0.1:34888 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260620 20:13:20.898455 28328 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:20.896781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34890 (local address 127.5.91.189:33389)
0620 20:13:20.896898 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.896903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.897450 (+ 547us) server_negotiation.cc:407] Connection header received
0620 20:13:20.897651 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.897655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.897711 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.897763 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.898337 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34890: BlockingRecv error: recv got EOF from 127.0.0.1:34890 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260620 20:13:20.918002 28328 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:20.916858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34898 (local address 127.5.91.189:33389)
0620 20:13:20.916976 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.916980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.917220 (+ 240us) server_negotiation.cc:407] Connection header received
0620 20:13:20.917349 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.917352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.917399 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.917464 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.917883 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34898: BlockingRecv error: recv got EOF from 127.0.0.1:34898 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260620 20:13:20.953220 28328 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:20.952049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34900 (local address 127.5.91.189:33389)
0620 20:13:20.952217 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:20.952220 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:20.952286 (+ 66us) server_negotiation.cc:407] Connection header received
0620 20:13:20.952401 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:20.952404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:20.952452 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:20.952546 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:20.953103 (+ 557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34900: BlockingRecv error: recv got EOF from 127.0.0.1:34900 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260620 20:13:21.020411 28328 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:21.019270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34910 (local address 127.5.91.189:33389)
0620 20:13:21.019435 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:21.019440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:21.019587 (+ 147us) server_negotiation.cc:407] Connection header received
0620 20:13:21.019739 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:21.019742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:21.019793 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:21.019859 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:21.020295 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34910: BlockingRecv error: recv got EOF from 127.0.0.1:34910 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260620 20:13:21.152774 28328 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:21.151247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34918 (local address 127.5.91.189:33389)
0620 20:13:21.151418 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:21.151421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:21.151581 (+ 160us) server_negotiation.cc:407] Connection header received
0620 20:13:21.151834 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:21.151838 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:21.151917 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:21.152038 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:21.152651 (+ 613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34918: BlockingRecv error: recv got EOF from 127.0.0.1:34918 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260620 20:13:21.412206 28328 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 20:13:21.411205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34932 (local address 127.5.91.189:33389)
0620 20:13:21.411333 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:21.411338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:21.411370 (+ 32us) server_negotiation.cc:407] Connection header received
0620 20:13:21.411536 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:21.411538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:21.411590 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:21.411662 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 20:13:21.412110 (+ 448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34932: BlockingRecv error: recv got EOF from 127.0.0.1:34932 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 20:13:21.699321 28432 catalog_manager.cc:1259] Loaded cluster ID: 8c0237d7b2f84a36ae7eb3d16a07fc2f
I20260620 20:13:21.699406 28432 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6079bb11914247659fad00f0b13829b6: loading cluster ID for follower catalog manager: success
I20260620 20:13:21.701074 28432 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6079bb11914247659fad00f0b13829b6: acquiring CA information for follower catalog manager: success
I20260620 20:13:21.702267 28432 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6079bb11914247659fad00f0b13829b6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 20:13:21.726985 28455 catalog_manager.cc:1259] Loaded cluster ID: 8c0237d7b2f84a36ae7eb3d16a07fc2f
I20260620 20:13:21.727074 28455 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 87e3effbf74c4af98f91a4caed3c5e02: loading cluster ID for follower catalog manager: success
I20260620 20:13:21.728711 28455 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 87e3effbf74c4af98f91a4caed3c5e02: acquiring CA information for follower catalog manager: success
I20260620 20:13:21.730121 28455 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 87e3effbf74c4af98f91a4caed3c5e02: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 20:13:21.930405 28529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:21.926251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34936 (local address 127.5.91.189:33389)
0620 20:13:21.926466 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:21.926469 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:21.926485 (+ 16us) server_negotiation.cc:407] Connection header received
0620 20:13:21.926667 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:21.926670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:21.926797 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:21.926896 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:13:21.927258 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:21.927820 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:21.928743 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:21.928974 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:21.929059 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:21.929407 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:13:21.929858 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:13:21.929952 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:21.930173 (+ 221us) server_negotiation.cc:299] Negotiation successful
0620 20:13:21.930243 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":69,"threads_started":1}
I20260620 20:13:21.937341 28535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:21.933584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56832 (local address 127.5.91.188:41955)
0620 20:13:21.933857 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:21.933860 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:21.933873 (+ 13us) server_negotiation.cc:407] Connection header received
0620 20:13:21.933948 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:21.933950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:21.934067 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:21.934215 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:13:21.934400 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:21.934924 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:21.935840 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:21.936066 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:21.936165 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:21.936476 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:13:21.936918 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:13:21.937022 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:21.937139 (+ 117us) server_negotiation.cc:299] Negotiation successful
0620 20:13:21.937190 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":91,"threads_started":1}
I20260620 20:13:21.944686 28535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:21.941429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56838 (local address 127.5.91.188:41955)
0620 20:13:21.941529 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:21.941533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:21.941547 (+ 14us) server_negotiation.cc:407] Connection header received
0620 20:13:21.941677 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:21.941679 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:21.941720 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:21.941783 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:13:21.941995 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:21.942703 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:21.943600 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:21.943913 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:21.944005 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:21.944224 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:13:21.944295 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:13:21.944355 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:21.944488 (+ 133us) server_negotiation.cc:299] Negotiation successful
0620 20:13:21.944542 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 20:13:21.945042 28529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:21.941111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34940 (local address 127.5.91.189:33389)
0620 20:13:21.941226 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:21.941229 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:21.941416 (+ 187us) server_negotiation.cc:407] Connection header received
0620 20:13:21.941658 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:21.941662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:21.941720 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:21.941787 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:13:21.942078 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:21.942758 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:21.943891 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:21.944239 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:21.944343 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:21.944579 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:13:21.944662 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:13:21.944705 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:21.944861 (+ 156us) server_negotiation.cc:299] Negotiation successful
0620 20:13:21.944917 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 20:13:21.945886 28542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 20:13:21.941066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36312 (local address 127.5.91.190:46445)
0620 20:13:21.941535 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0620 20:13:21.941540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 20:13:21.941999 (+ 459us) server_negotiation.cc:407] Connection header received
0620 20:13:21.942093 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 20:13:21.942096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 20:13:21.942276 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 20:13:21.942392 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 20:13:21.942609 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:21.943380 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:21.944381 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 20:13:21.944690 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 20:13:21.944788 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 20:13:21.945047 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 20:13:21.945389 (+ 342us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 20:13:21.945482 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 20:13:21.945694 (+ 212us) server_negotiation.cc:299] Negotiation successful
0620 20:13:21.945750 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":74,"threads_started":1}
I20260620 20:13:21.948886 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 28179
I20260620 20:13:21.954380 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 28254
I20260620 20:13:21.959676 5486 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSTSRqO/build/debug/bin/kudu with pid 28330
2026-06-20T20:13:21Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3068 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6099 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (290813 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST